Create a Dynamic Sitemap for Your Next.js Blog

Sitemaps are documents that can be leveraged to help web crawlers analyze your website/blog more effectively.

They are especially useful for large sites with lots of content. Read more about sitemaps to see if they're right for your use case

Sitemap Structure

Sitemaps are most commonly written in XML.

Generally they contain an xml tag to declare versioning and encoding, and a urlset to list out site URLs:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.noahmatsell.ca/</loc>
  </url>
  <url>
    <loc>https://www.noahmatsell.ca/blog</loc>
  </url>
</urlset>

Site URLs are declared using a child url tag. Inside each url tag, you need at minimum a loc child tag which declares the URL. Other optional child tags for a url entry include lastmod, changefreq, and priority. Read more about these tags here.

Building Script

Adding entries to a sitemap manually is perfectly valid, but it can become tedious. Especially if you have a really large site — like a blog with many posts. Manual entry can also be error prone, and relies on you remembering to make updates every time a new page is created.

The problem of list and updating URLs for a sitemap is a great candidate for automation. The following will help you automate the process of creating a sitemap.xml file for your website, and populate this file with all of the URLs desired.

1. Create script file

Create build-sitemap.ts script, that we will populate in the following steps:

// build-sitemap.ts
async function generate() {
  ...
}

generate();

2. Define Your Dynamic Pages

Inside the generate function, we start by getting an array all posts with their slugs and publish date:

// build-sitemap.ts
...
const allPosts = getAllPosts(["slug", "publishDate"])
  .filter((post) => {
    return Boolean(post.publishDate && post.slug);
  })
  .map((post) => {
    return { ...post, slug: `blog/${post.slug}` };
  }) as PageEntry[];
...

This code takes advantage of my blog's getAllPosts function, which reads all of my .md files and returns the requested metadata fields (in this case slug and publishDate). You can learn more about setting this up in my blog post here, or jump straight to the code here.

3. Define Your Static Pages

Next, we can manually build a list of any other URLs we want to include:

// build-sitemap.ts
const staticPages = [{ slug: "" }, { slug: "resume" }, { slug: "blog" }];

Alternatively, if you can use a tool like globby to get all pages in specified folders using pattern matching

• First install globby using npm or yarn
• Then define the glob pattern to find your pages:

...
const staticPages = await globby([
  'pages/*.tsx',
  '!pages/_*.tsx',
  '!pages/api',
  '!pages/404.tsx'
]);
...

• This pattern will find all of your pages under the pages directory, except for those that have a _ prefix, your 404 page, and you pages under the api route.

I personally don't use globby because I only have a few static pages that I want to include in my sitemap.

4. Generate XML Markup

Create a generateSiteMap function that will generate the XML markup for your sitemap. Provided an array of page entries, it will loop through the pages and build up a list of child url tags under the urlset tag.

// build-sitemap.ts
const ROOT_URL = "https://www.noahmatsell.ca";

interface PageEntry {
  slug: string;
  publishDate?: string;
}

function generateSiteMap(pages: PageEntry[]) {
  return `<?xml version="1.0" encoding="UTF-8"?>
   <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     ${pages
       .map(
         ({ slug, publishDate }) => `
          <url>
              <loc>${`${ROOT_URL}/${slug}`}</loc>
              ${publishDate ? `<lastmod>${publishDate}</lastmod>` : ``}
          </url>`
       )
       .join("")}
   </urlset>
 `;
}

Now inside of the generate function, I can call this function with my previously defined page arrays:

// build-sitemap.ts
...
const sitemap = generateSiteMap([...staticPages, ...allPosts]);
...

5. Format and Write File

Finally, still in the generate function nwe can format this file content using prettier (this is optional), and write it to our filesystem:

// build-sitemap.ts
...
const prettierConfig = await prettier.resolveConfig("./.prettierrc.js");

const formatted = prettier.format(sitemap, {
    ...prettierConfig,
    parser: "html",
  });

writeFileSync("public/sitemap.xml", formatted);
...

And that's it. You have a script that will dynamically create your sitemap. It will be written to the public folder, and become a live resource when deployed.

Review full code here

Triggering Sitemap Build

Now that we have our script, we can run it manually any time from the terminal. My preferred method is using ts-node like:

npx ts-node ./lib/build-sitemap.ts

However to fully automate our sitemap generation, we can create a postbuild script in your project's package.json file:

"scripts": {
  "dev": "next dev",
  "build": "next build",
  "postbuild": "npx ts-node ./lib/build-sitemap.ts",
  ...

The beauty of postbuild is that it will automatically run any time your site is built for deployment. This means that the latest changes to your site will be reflected automatically in your sitemap.xml file.

Wrapping Up

Give this a try on your Next.js website. Once you've written and deployed this script successfully, you can check out your sitemap live at yourdomain.com/sitemap.xml. Ensure that Google always has the latest version of your site by adding this sitemap via the Google Search Console.