Token Optimization

Reduce token count in converted Markdown by 10-30% for cheaper LLM inference, RAG pipelines, and agentic workflows.

Token Optimization post-processes converted Markdown to reduce its token count without changing the meaning or structure of the content. This is designed for teams feeding converted documents into LLM pipelines, where every token has a cost.

What It Does

Token optimization applies structural transformations to the Markdown output:

  • Removes redundant whitespace and formatting artifacts.
  • Normalizes heading levels and list structures.
  • Collapses unnecessary line breaks.
  • Strips extraneous metadata that does not contribute to content meaning.

These are purely structural optimizations. The semantic content of your document is preserved exactly.

Token optimization does not summarize, truncate, or alter the meaning of your content. It only removes structural overhead.

Availability

Token optimization is available on all paid plans (Starter, Professional, and Business). It does not cost additional credits — there is no surcharge beyond the standard or Enhanced AI conversion cost.

Response Fields

When token optimization is enabled, the API response includes a token_optimization object with savings statistics:

{
    "success": true,
    "markdown": "# Optimized Document\n\nContent here...",
    "credits_used": 1,
    "credits_remaining": 499,
    "token_optimization": {
        "original_chars": 15420,
        "optimized_chars": 12180,
        "savings_percent": 21.01
    }
}
FieldTypeDescription
original_charsintegerCharacter count before optimization.
optimized_charsintegerCharacter count after optimization.
savings_percentfloatPercentage reduction in character count.

Enabling Token Optimization

Toggle Token Optimizer on before clicking Convert.

Pass optimize_tokens: true in your conversion request:

curl -X POST https://markdownanything.com/api/v1/convert \
  -H "Authorization: Bearer mda_your_token" \
  -F "[email protected]" \
  -F "optimize_tokens=true"

Go to Settings > Workspace and enable Prefer Token Optimization. All future conversions in this workspace will optimize tokens by default unless overridden per-request.

Use Cases

Token optimization is most valuable when:

  • Feeding documents to LLMs — Reduce input tokens to lower API costs with OpenAI, Anthropic, or other providers.
  • Building RAG pipelines — Smaller chunks mean more efficient embedding and retrieval.
  • Batch processing — Savings compound across thousands of documents.

Typical savings range from 10-30% depending on the source format and document structure. PDFs and Word documents with heavy formatting tend to see the largest reductions.

Combining with Enhanced AI

Token optimization works alongside Enhanced AI. When both are enabled, the pipeline is:

  1. File is converted to Markdown (with AI-powered extraction if Enhanced AI is on).
  2. Token optimization post-processes the output.
  3. The optimized result is returned with savings statistics.

The credit cost is determined by the conversion type (standard or Enhanced AI) — token optimization adds no extra credit charge.

Next Steps