Legacy HTML normalization: replacing <p> with <div> on import

When migrating old content, another conflict between legacy WYSIWYG markup and the new React/SSR rendering was discovered. The old editor wrapped large HTML fragments in <p>. In the new version, such imported HTML can end up inside an already existing paragraph, causing React to receive an invalid HTML structure and warn:

In HTML, <p> cannot be a descendant of <p>. This will cause a hydration error.

Cause

The issue is not with the specific text, but with the semantics of the <p> tag: HTML does not allow nested paragraphs. Legacy WYSIWYG used <p> as a universal block wrapper, although a neutral container like <div> is safer for arbitrary nested content.

This is especially unpleasant during SSR: the browser may automatically correct the invalid HTML structure differently than React expects, and the resulting DOM structure after parsing differs from the server markup. This leads to a hydration mismatch/error.

Decision Made

A cheerio helper has been added to the importer, which replaces all <p> tags with <div> before saving legacy HTML, preserving the inner HTML and attributes:

import * as cheerio from 'cheerio'

export function replacePWithDiv(html: string): string {
  const $ = cheerio.load(html, { xml: false }, false)

  $('p').each((_, el) => {
    const $el = $(el)
    const div = $('<div></div>')
    div.html($el.html() || '')

    for (const attr of el.attributes) {
      div.attr(attr.name, attr.value)
    }

    $el.replaceWith(div)
  })

  return $.html()
}

Why normalization is performed in the importer

This is a legacy-specific source data issue, so it is safer to fix it at the entry point to the new system rather than forcing the general Markdown/React renderer to constantly compensate for the old WYSIWYG markup.

Here, the importer acts as a normalization layer: it preserves the meaning of the old content, but removes the structure that is known to conflict with valid HTML and React hydration.

Significance for the general Markdown task

This case complements the whitespace/indentation problem: the new pipeline must take into account not only Markdown syntax, but also the quality of embedded HTML. Even if the Markdown is formally the same, legacy HTML within it can have a structure that different parser/renderer chains process differently.

The local solution remains part of the BiznesHelper compatibility layer until a more general normalization/validation pipeline appears in haih-agent.

New case: legacy HTML normalization before MDX/Markdown rendering

When importing content from old CKEditor, a new category of incompatibility emerged: even after eliminating nested <p> tags, some HTML remains syntactically valid for the browser, but survives MDX parsing and subsequent SSR rendering poorly.

Symptoms

The MDX parser crashed with errors like:

Error parsing markdown: Expected the closing tag `</p>` either after the end of `paragraph` ...
Error parsing markdown: Expected the closing tag `</div>` either after the end of `paragraph` ...

A typical legacy fragment looked like this:

<p style="text-align:justify"><img alt="" src="/ckeditor_assets/pictures/40/content_oil.png" style="border-style:solid; border-width:1px; height:503px; width:703px"><br>
<span style="font-family:trebuchet ms,helvetica,sans-serif; font-size:14px">Text...</span></p>

Here, several problems are combined simultaneously:

• <br> is written in HTML form, but the MDX/JSX pipeline expects a self-closing variant;
• the old WYSIWYG actively uses <p> as a universal container;
• line breaks and whitespace between HTML tags can be interpreted by MDX no longer as neutral formatting of the source, but as boundaries of markdown paragraphs.

What was tested

Initially, normalization via htmlparser2 + dom-serializer was considered, but this option turned out to be inconvenient for the task: self-closing tags were not normalized the way MDX requires, and text could be serialized with undesirable HTML entities.

A working option turned out to be rehype, which allows first bringing legacy HTML to a more stable serialization, and then passing the result to the general Markdown normalization pipeline.

Current pipeline

Normalization now consists of several sequential steps.

1. Legacy <p> are replaced with <div>

This eliminates invalid paragraph nesting and reduces the risk of hydration mismatch:

function replacePWithDiv(html: string): string {
  const $ = cheerio.load(html, { xml: false }, false)

  $('p').each((_, el) => {
    const $el = $(el)
    const div = $('<div></div>')

    div.html($el.html() || '')

    for (const attr of el.attributes) {
      div.attr(attr.name, attr.value)
    }

    $el.replaceWith(div)
  })

  return $.html()
}

2. HTML is passed through rehype

rehype is used as a parser/stringifier to fix and canonize legacy markup:

const normalized = await rehype()
  .data('settings', {
    fragment: true,
    closeSelfClosing: true,
  })
  .process(html)

The key effect is that self-closing elements like <br> are brought into a form that passes through MDX/JSX processing more safely.

3. Whitespace between adjacent tags is removed

After serialization, line breaks and spaces between HTML tags are additionally collapsed:

html = String(normalized).replace(/>\s+</g, '><')

This is important specifically for MDX: a line break between inline/block HTML nodes can affect markdown parsing and generate an unexpected paragraph boundary. In browser HTML, such whitespace is often irrelevant, but in a mixed Markdown+HTML format, it is no longer the case.

Final implementation

The logic is gathered into the cleanupOldContent helper, which works as a separate normalization layer between legacy CKEditor HTML and the new Concept/Markdown system:

import * as cheerio from 'cheerio'
import { rehype } from 'rehype'
import { normalizeMarkdownContent } from '../../KBConcept/helpers/normalizeMarkdownContent'

function replacePWithDiv(html: string): string {
  const $ = cheerio.load(html, { xml: false }, false)

  $('p').each((_, el) => {
    const $el = $(el)
    const div = $('<div></div>')
    div.html($el.html() || '')

    for (const attr of el.attributes) {
      div.attr(attr.name, attr.value)
    }

    $el.replaceWith(div)
  })

  return $.html()
}

export async function cleanupOldContent(html: string): Promise<string> {
  html = html.replaceAll(/<u>([^<]+)?<\/u>:/g, '<u>$1:</u>')
  html = replacePWithDiv(html)

  try {
    const normalized = await rehype()
      .data('settings', {
        fragment: true,
        closeSelfClosing: true,
      })
      .process(html)

    html = String(normalized).replace(/>\s+</g, '><')

    return await normalizeMarkdownContent(html)
  } catch (error) {
    console.error(error)
    console.error('cleanupOldContent html', html)
    throw error
  }
}

Architectural conclusion

This case shows that the local compatibility layer for BiznesHelper must normalize not only Markdown whitespace, but also embedded legacy HTML before content hits the MDX/SSR pipeline.

That is, the actual import path now looks like this:

legacy CKEditor HTML
→ DOM-level cleanup
→ HTML normalization via rehype
→ whitespace normalization
→ general normalizeMarkdownContent
→ Concept saving

This is a useful separation of concerns: browser-tolerant, but unstable legacy markup is fixed once at the entrance, rather than endlessly compensated for in the public renderer.

Status

The specific case with line breaks between tags and self-closing HTML is now closed by a local normalization helper. The general compatibility issue of MDXEditor / Markdown renderer remains a subject of a separate R&D task in haih-agent.