Skip to main content

​
Overview

IndexNow is an open protocol that lets you instantly notify search engines whenever a URL on your site is created, updated, or deleted. Instead of waiting for crawlers to find your changes, Sight AI pushes the URL straight to Bing’s IndexNow endpoint — and Bing relays it to every other participating engine. Once IndexNow is set up, Sight AI handles submissions for you automatically:
  • New URLs from your sitemap are submitted as soon as we detect them
  • Articles you publish through Sight AI are submitted the moment they sync to your CMS
  • Updated URLs are re-submitted when they change
Google does not participate in IndexNow. Use the Google Search Console integration for Google indexing. Run both for full search-engine coverage.

​
Participating search engines

A single IndexNow submission reaches all of these:
  • Bing — Microsoft’s search engine
  • Yandex — Russia’s largest search engine
  • Seznam — Czech Republic’s search engine
  • Naver — South Korea’s largest search engine

​
How verification works

Bing won’t accept submissions on behalf of a domain unless it can prove you own that domain. The proof is simple: you host a small text file at the root of your site that contains a unique key, and Bing fetches it whenever it accepts a submission. The Sight AI flow is:
1

Generate a key in Sight AI

A 32-character alphanumeric key is created for your site. This becomes both the file name and the file contents.
2

Host the key file at your site root

The file must be reachable at https://yourdomain.com/<your-key>.txt and return the key as plain text.
3

Sight AI verifies the file is reachable

We fetch both the apex (yourdomain.com) and www.yourdomain.com variants. As long as one of them returns the exact key, you’re verified.
4

Sight AI starts submitting URLs automatically

From now on every sitemap change and every published article triggers an IndexNow ping. No further action required.

​
Before you start

You’ll need three things — Bing Webmaster Tools is the one most people forget, so we cover it first.

A Bing Webmaster Tools account with your site added

Free, takes ~2 minutes. Required by Bing — see the step-by-step below.

Your site already added to Sight AI

The IndexNow setup lives at Integrations → Bing IndexNow and operates on the active workspace.

A way to upload a file to your site root

Or, on platforms that block root uploads, the ability to add a redirect from /<key>.txt to a hosted asset.

​
Prerequisite: Bing Webmaster Tools

Bing won’t accept IndexNow submissions for a domain until that domain is registered and verified inside Bing Webmaster Tools — it’s a separate Microsoft product from Sight AI, and there’s no way to skip it. The good news is the setup is short:
1

Create a free Bing Webmaster Tools account

Go to bing.com/webmasters and sign in with a Microsoft, Google, or Facebook account. If you already have a Microsoft account (Outlook, Hotmail, Xbox, etc.) you can use that.
2

Add your site and verify ownership

Once you’re in, click Add a site in the top-left dropdown and enter your full site URL (with https:// and the correct www/non-www variant). Bing offers four ways to verify ownership — XML file, meta tag, CNAME DNS record, or auto-import from Google Search Console (the easiest, if you already have GSC connected).Microsoft’s official walkthrough covers every verification method in detail: Add and Verify your site — Bing Webmaster Tools.
If you’ve already connected Google Search Console (see the GSC integration guide), use the “Import from Google Search Console” option in Bing — it pulls the property over and verifies it for you in one click. No file uploads or DNS changes needed.
3

Confirm the site shows as verified in Bing Webmaster Tools

A green checkmark next to your domain in the left-hand site picker means you’re done. Now you can come back to Sight AI and start the IndexNow setup.
Bing Webmaster Tools is a Microsoft product — Sight AI cannot help with issues inside it. If you get stuck creating the account, verifying ownership, or anything else inside Bing Webmaster Tools, those questions need to go to Microsoft. The right channels are:Once your site shows as verified inside Bing Webmaster Tools, everything below this point is something we can absolutely help with — email support@trysight.ai.

​
Setup by site type

In Sight AI, open Integrations → Bing IndexNow, pick your workspace, and select the path that matches your site:
The Sight AI WordPress plugin handles IndexNow for you. If you’ve already installed the plugin (see the WordPress integration guide), pick WordPress in the IndexNow setup, generate a key, and the plugin will host the file automatically.If you’re hosting WordPress yourself and want to do it manually:
1

Download the key file from Sight AI

It will be named <your-key>.txt and contain only the key.
2

Upload it to your WordPress site root

Use FTP/SFTP, cPanel File Manager, or a plugin like WP File Manager. The file must sit at the same level as wp-config.php and index.php — not inside /wp-content, /wp-admin, or any other subdirectory.
3

Confirm it's reachable

Open https://yourdomain.com/<your-key>.txt in your browser. You should see the key as plain text — nothing else.
4

Click Verify in Sight AI

Once verified, Sight AI starts pinging IndexNow on your behalf.

​
Custom site setup

This section is the deep dive for any site that isn’t WordPress or Webflow. The goal is the same in every case: make https://yourdomain.com/<your-key>.txt return your key as plain text.
Make sure you’ve completed the Bing Webmaster Tools prerequisite above before starting. If your domain isn’t verified in BWT, IndexNow submissions will silently fail no matter how perfect the key file is. The setup is just two steps: create a free account at bing.com/webmasters and follow Microsoft’s “Add and Verify your site” guide.

​
Step 1 — Generate the key in Sight AI

  1. Open Integrations → Bing IndexNow.
  2. Pick your workspace.
  3. Select All other sites.
  4. Confirm you’ve registered with Bing Webmaster Tools.
  5. Click Generate IndexNow key. Sight AI creates a 32-character key for you.
  6. Click Download key file. You’ll get a file like aB3xZ9k7…q2P.txt whose contents are just the 32-character key — nothing else.
The file must contain ONLY the key. No leading/trailing spaces, no blank lines, no <html> wrapper, no UTF-8 BOM. Bing rejects files that don’t byte-for-byte match the key.

​
Step 2 — Understand “site root”

“Site root” means the file is reachable directly under your domain — not under any path segment. Test against this rule:
URLAt root?
https://example.com/abc123.txt✅ Yes
https://example.com/abc123.txt/ (trailing slash)❌ No
https://example.com/static/abc123.txt❌ No
https://example.com/public/abc123.txt❌ No
https://example.com/.well-known/abc123.txt❌ No
https://example.com/blog/abc123.txt❌ No
If you can serve a robots.txt or sitemap.xml from your site, you can serve the IndexNow key file the same way and at the same place.

​
Step 3 — Host the file (pick your platform)

Drop the file into your project’s public/ directory:
your-project/
  public/
    abc123…q2P.txt   ← your downloaded file goes here
Anything in public/ is served from the root, so after deploying it’ll be reachable at https://yourdomain.com/abc123…q2P.txt.Commit, push, wait for the deploy to finish, then click Verify in Sight AI.
In the Pages Router, public/ works the same way. In the App Router, this is still the right place — don’t put it inside app/.
Add the file to whichever directory Netlify publishes from (commonly public/, dist/, build/, or _site/). Check Site settings → Build & deploy → Continuous deployment → Publish directory if you’re not sure.For a static site:
your-site/
  public/
    abc123…q2P.txt
Deploy the site, then verify in Sight AI.If you can’t drop a file into the build, you can instead add a redirect to _redirects:
/abc123…q2P.txt    https://your-cdn.example.com/abc123…q2P.txt    200
The 200 rewrite (not a 301) keeps the URL as yourdomain.com/... so Bing accepts it.
Cloudflare Pages serves the contents of your build output directly. Add the key file alongside your other static assets in the directory that becomes the site root (often public/, dist/, or the project root for fully static sites). Push and let Pages rebuild.To verify locally before deploying, run wrangler pages dev and hit http://localhost:8788/<key>.txt.
  1. In your repository, drop the <key>.txt file at the root of the branch GitHub Pages serves from (usually main or gh-pages, in the directory configured under Settings → Pages).
  2. Commit and push.
  3. Wait for the Pages build to finish (check the Actions tab).
  4. Verify by opening https://your-username.github.io/<key>.txt (or your custom domain).
If you use Jekyll, Jekyll will ignore files starting with _. Your key won’t start with _, but if you also keep a .nojekyll file at the root, you’re guaranteed the file will be served as-is.
Static-site generators all expose a folder whose contents are copied to the site root unchanged:
GeneratorDrop the file in
Astropublic/
Hugostatic/
Jekyllsite root (any file with no front matter)
Eleventythe directory listed under passthroughCopy (often public/ or root)
Gatsbystatic/
Next.jspublic/
SvelteKitstatic/
Nuxtpublic/
Rebuild and deploy, then verify in Sight AI.
  1. Use FTP/SFTP, cPanel File Manager, or scp to upload <key>.txt to your DocumentRoot — the same directory that contains your index.html or index.php.
  2. Make sure the file’s permissions are 644 so the web server can read it (chmod 644 <key>.txt).
  3. If you have an aggressive .htaccess rewrite that routes everything through a single PHP entrypoint, add an exception so the .txt file is served directly: 
    RewriteEngine On
RewriteRule ^[a-zA-Z0-9-]+\.txt$ - [L]
    Place this above your existing rewrite rules.
  4. Verify in your browser, then click Verify in Sight AI.
Upload <key>.txt to your site’s web root (whatever root points to in your server block).If your config has a SPA-style catch-all like try_files $uri $uri/ /index.html, that catch-all will swallow the .txt request and return your HTML page. Add an explicit location block above the catch-all so the key file is served as-is:
location ~* ^/[a-zA-Z0-9-]+\.txt$ {
  add_header Content-Type text/plain;
  try_files $uri =404;
}
Reload Nginx (sudo nginx -s reload) and verify.
  1. Copy <key>.txt into the site’s physical directory (the one mapped under Sites → your-site → Basic Settings → Physical path).
  2. In IIS Manager → MIME Types, confirm .txt is mapped to text/plain (it is by default).
  3. If your web.config rewrites everything to a single handler, add a precondition above the rewrite: 
    <rule name="IndexNowKey" stopProcessing="true">
  <match url="^[a-zA-Z0-9-]+\.txt$" />
  <action type="None" />
</rule>
  4. Verify in your browser, then in Sight AI.
  1. Upload <key>.txt to the S3 bucket that backs your CloudFront distribution. Place it at the bucket root, not inside any prefix.
  2. Set the object’s Content-Type to text/plain (the upload UI lets you set this; via CLI: --content-type text/plain).
  3. Make sure the object is publicly readable, or that your bucket policy allows the CloudFront OAI/OAC to read it.
  4. If CloudFront has aggressive caching for .txt files, invalidate the path: /<key>.txt.
  5. Verify the URL responds, then click Verify in Sight AI.
For Workers Sites, drop the file into the directory listed under [site] bucket in wrangler.toml (commonly public/).For a hand-written Worker, add an early route that returns the key as plain text:
export default {
  async fetch(request) {
    const url = new URL(request.url);
    if (url.pathname === "/abc123…q2P.txt") {
      return new Response("abc123…q2P", {
        headers: { "Content-Type": "text/plain" },
      });
    }
    // …rest of your worker
  },
};
Replace both occurrences of abc123…q2P with your actual key.
Framer doesn’t let you upload arbitrary files to the site root, so the cleanest path is to use the Sight AI Framer plugin which handles indexing for you (see the Framer integration guide).If you don’t want to use the plugin, you can use a Custom Code embed plus a redirect through your DNS provider, but in practice we recommend the plugin — IndexNow setup is one of the things it’s designed to manage.
On a self-hosted Ghost install, place <key>.txt in content/files/ (it will be served at https://yourdomain.com/content/files/<key>.txt, which is not the root) — so instead use a small Express/Nginx reverse-proxy rule above Ghost:
location ~* ^/[a-zA-Z0-9-]+\.txt$ {
  alias /var/www/ghost/content/files/$1.txt;
}
For Ghost Pro (managed hosting), root file uploads aren’t supported. Contact Ghost support to host the key file or move IndexNow management into Cloudflare in front of Ghost using a Worker (see Cloudflare Workers above).
Shopify doesn’t allow root file uploads on the storefront, but you can serve the key file through a redirect:
  1. In Shopify admin, go to Online Store → Navigation → URL Redirects.
  2. Click Create URL redirect.
  3. Redirect from: /<your-key>.txt
  4. Redirect to: a public URL where the file lives (e.g., a Files entry — upload the .txt through Settings → Files, copy the CDN URL, paste it here).
  5. Save, then verify in Sight AI.
Shopify converts redirects to 301s. Sight AI follows redirects during verification, and Bing’s verifier follows them too, so this is supported.
Squarespace doesn’t expose root file uploads on most plans. Two options:
  1. Code Injection (Business plan and above): doesn’t help here because Code Injection runs inside HTML pages, not as a standalone file. Skip this option.
  2. URL redirects: Squarespace lets you create URL mappings under Settings → Advanced → URL Mappings. The syntax is /<your-key>.txt -> https://your-cdn.example.com/<your-key>.txt 301. Host the file on any public URL (S3, GitHub Pages, etc.) and redirect to it.
If neither option works on your plan, IndexNow setup isn’t possible on Squarespace — Sight AI’s Google Search Console integration still works as a fallback.
Wix doesn’t allow custom files at the site root and doesn’t support arbitrary 301 redirects to external URLs for non-Wix paths, so IndexNow setup typically isn’t possible on Wix.If your site is on Wix Studio with a custom domain, you can sometimes use the Wix CLI to upload to the root — contact Wix support to confirm your plan supports it.For most Wix users, fall back to the Google Search Console integration — it doesn’t require any file hosting.
Add a single early route that returns the key as plain text. Example for Express:
app.get("/abc123…q2P.txt", (req, res) => {
  res.type("text/plain").send("abc123…q2P");
});
Make sure this route is registered before any catch-all middleware (404 handler, SPA fallback, etc.). Replace both occurrences with your actual key.Same idea for FastAPI:
from fastapi.responses import PlainTextResponse

@app.get("/abc123…q2P.txt", response_class=PlainTextResponse)
def indexnow_key():
    return "abc123…q2P"

​
Step 4 — Confirm the file is reachable

Before clicking Verify in Sight AI, sanity-check the file from your terminal: 
curl -i https://yourdomain.com/<your-key>.txt
You want a response that looks like this: 
HTTP/2 200
content-type: text/plain
content-length: 32

aB3xZ9k7…q2P
The exact content-type doesn’t have to be text/plain — application/octet-stream and others work too — but the status must be 200 and the body must be exactly your key with no extras. Run the same check on the www variant if your site uses it: 
curl -i https://www.yourdomain.com/<your-key>.txt
Sight AI tries both during verification, so you only need one of them to work.

​
Step 5 — Verify in Sight AI

Back in Integrations → Bing IndexNow, click Verify connection. Sight AI will:
  1. Fetch https://yourdomain.com/<your-key>.txt
  2. If that fails, try https://www.yourdomain.com/<your-key>.txt
  3. Compare the response body (after trimming whitespace) to your key
  4. On success: mark the integration as verified, immediately submit your existing sitemap URLs, and start pinging IndexNow on every future change
If verification fails, jump to the Troubleshooting section — it covers every error message we surface.

​
After verification

Once verified, IndexNow runs entirely in the background:
  • New sitemap URLs — picked up by Sight AI’s hourly sitemap check and submitted as a batch
  • New articles — submitted the moment they sync to your CMS
  • Updated content — re-submitted when the URL appears again in your sitemap with a new lastmod
  • No rate limits to worry about — Sight AI batches up to 10,000 URLs per IndexNow request and Bing accepts them all
  • No further action required from you — the key file just needs to stay reachable
You can review every submission under Indexing Activity.
Leave the key file in place forever. Bing periodically re-verifies ownership by re-fetching the file. If you delete it, future submissions will start failing silently.

​
Troubleshooting

When verification fails, Sight AI returns one of the messages below. Each section maps the message to its likely cause and the exact fix.

​
”Key file not accessible — HTTP 404”

The most common failure. Bing tried to fetch https://yourdomain.com/<your-key>.txt and your server returned 404. Causes and fixes:
  • The file isn’t at the root. Check curl -I https://yourdomain.com/<your-key>.txt — if you get 404, but a path like https://yourdomain.com/static/<your-key>.txt returns 200, you uploaded it to the wrong directory. Move it to the root.
  • Your build process didn’t include the file. If you committed it to source but didn’t push, or your build folder doesn’t include the static files directory, the file won’t ship. Confirm the file is in the deployed output (open the URL in a browser tab — incognito to avoid caching).
  • A SPA catch-all is swallowing the request. If you ship a single-page app, your server is probably returning index.html for every unknown path. The text response will look like HTML, not your key. See the Apache/Nginx/IIS sections above for how to add a precondition that lets .txt files through.
  • You’re testing the wrong host. Sight AI tries both apex and www. Check both manually.

​
”Key file content mismatch”

Sight AI fetched the file successfully but the body didn’t match your key. Causes and fixes:
  • The file has extra characters. Open the file in a hex-aware editor (xxd <key>.txt | head on macOS/Linux). The bytes should be only [a-zA-Z0-9] — no spaces, no newlines, no UTF-8 BOM (ef bb bf at the start).
  • You uploaded the wrong file. If you regenerated the key in Sight AI and forgot to re-download, your hosted file is stale. Click Download key file again and re-upload.
  • A CDN is rewriting the response. Some CDNs minify or wrap text responses. Disable rewrite rules for this path or set caching to bypass.
  • Your server returned an error page with HTTP 200. Some hosts return a “soft 404” — a 200 status with an HTML error body. Check the actual body of the response (not just the status code).

​
”DNS resolution failed — the domain could not be found”

Sight AI couldn’t resolve the host at all. Causes and fixes:
  • The site host stored on your workspace doesn’t match a real domain. Open Site Settings → General and confirm the domain is correct.
  • The domain just changed nameservers and DNS hasn’t propagated. Wait 15–30 minutes and try again.
  • A typo: example.con vs example.com.

​
”TLS/SSL certificate error”

Sight AI reached the server but couldn’t establish a secure connection. Causes and fixes:
  • Your SSL certificate is expired or doesn’t cover the host you’re verifying (e.g., the cert covers example.com but you’re verifying www.example.com).
  • Your server is using an outdated cipher Sight AI doesn’t accept. Renew the cert from your hosting provider or run SSL Labs’ test to diagnose.

​
”Connection refused” / “Connection reset”

Your server actively rejected Sight AI’s request, usually because of a WAF rule or bot-detection layer. Causes and fixes:
  • Cloudflare Bot Fight Mode / WAF: allow-list Sight AI’s verifier in Cloudflare → Security → WAF by exempting requests for *.txt paths, or temporarily set Bot Fight Mode to off, verify, then re-enable.
  • Vercel firewall rules: add an allow rule for the <key>.txt path.
  • Server fail2ban / rate limiter: check your access logs for blocks on Sight AI’s verifier (User-Agent contains Mozilla/5.0).

​
”Request timeout — unable to access the key file”

Your server took longer than 10 seconds to respond. Causes and fixes:
  • The server is overloaded or cold-starting. Wait a few seconds and retry.
  • The file is being generated dynamically and that handler is slow. Serve it as a true static asset instead.

​
Verification succeeded but submissions are still failing

Verification passed but the Indexing Activity page shows submissions failing. Causes and fixes:
  • Your domain isn’t registered in Bing Webmaster Tools. This is the silent killer — Sight AI’s verification only checks that the key file is reachable; it doesn’t (and can’t) check whether you’ve also added the site to BWT. If you skipped the BWT prerequisite, Bing will accept the IndexNow request but discard it on its end. Go to bing.com/webmasters, add your site, and verify ownership using Microsoft’s guide.
  • You moved or deleted the key file after verifying. Bing re-verifies ownership periodically. Re-upload the file to the root.
  • The submitted URL host doesn’t match the verified host. If your sitemap returns https://www.example.com/page but you verified example.com (apex only), Bing rejects the submission. The fix is in the sitemap, not IndexNow — make your sitemap consistently use the same host you verified, or set the canonical host on your site so both work.
  • The page returns a non-2xx status. Bing’s crawler will follow the IndexNow ping but won’t index a URL that returns 4xx or 5xx. Check that the page actually loads.

​
How to ask for help

For anything inside Bing Webmaster Tools (creating the account, verifying ownership, settings inside BWT, indexing decisions Bing makes about your URLs), Sight AI can’t see what’s happening on Microsoft’s side — those go to Microsoft directly: For anything on the Sight AI side (verification fails, key file logic, submission status, sitemap-host mismatch, our integration UI), email support@trysight.ai with:
  • The exact error text from the verification panel
  • The output of curl -i https://yourdomain.com/<your-key>.txt
  • The output of curl -i https://www.yourdomain.com/<your-key>.txt
  • A screenshot of the Integrations → Bing IndexNow page
  • Confirmation that the domain shows as verified inside Bing Webmaster Tools
That’s almost always enough to diagnose in a single round-trip.

​
Key reference

For anyone bringing their own key (under Use my own key in the Sight AI UI), it must conform to the IndexNow spec:
RuleAllowed
Length8 to 128 characters
Allowed charactersa-z, A-Z, 0-9, - (hyphen)
Not allowedunderscores, dots, slashes, spaces, any other punctuation
File nameexactly <key>.txt (case-sensitive — keep the .txt lowercase)
File contentsexactly the key, no whitespace, no newline, no BOM
File locationsite root only
If you let Sight AI generate the key, it always meets these rules.

​
Disconnect or regenerate

You can manage the integration at any time from Integrations → Bing IndexNow:
  • Regenerate key — issues a new key and clears the verified flag. You’ll need to re-upload a new file and re-verify. Use this if your old key was leaked or you want to rotate it.
  • Use my own key — replace the auto-generated key with one you brought from elsewhere (e.g., one already used with another tool that submits to IndexNow).
  • Disconnect — removes the IndexNow key for the workspace, stops automatic submissions, and clears the verified flag. The file on your server is harmless to leave behind, but you can also delete it.

​
Next steps