JavaScript Tooling Internals: Code and Comment Parsing
acorn and esprima are JavaScript parsers that convert source code into Abstract Syntax Trees (ASTs) for analysis or transformation. doctrine is a utility specifically designed to parse JSDoc comments into structured data. jsdoc is a full-featured documentation generator that uses parsing logic to produce HTML documentation from code comments. Together, these packages form the backbone of many linters, bundlers, and documentation systems in the JavaScript ecosystem.
Npm Package Weekly Downloads Trend
Github Stars Ranking
Stat Detail
JavaScript Tooling Internals: Code and Comment Parsing
When building developer tools like linters, bundlers, or documentation generators, you need reliable ways to understand JavaScript code and its comments. acorn and esprima handle the code itself, turning it into a tree structure developers can analyze. doctrine and jsdoc focus on the comments, specifically JSDoc tags, to extract metadata or generate docs. Let's break down how they differ and where each fits in your architecture.
🧩 Parsing JavaScript Code: Acorn vs Esprima
Both acorn and esprima convert JavaScript source code into an Abstract Syntax Tree (AST). This tree represents the code structure, allowing tools to check for errors, optimize performance, or transform syntax.
acorn is known for being small, fast, and highly modular. It supports modern ECMAScript versions through plugins.
// acorn: Parsing modern JavaScript
const acorn = require("acorn");
const ast = acorn.parse("let x = 10;", {
ecmaVersion: 2020,
sourceType: "module"
});
console.log(ast.body[0].type); // "VariableDeclaration"
esprima was one of the first widely adopted JavaScript parsers. It is stable and produces a standard AST, but it is generally slower and less active than Acorn.
// esprima: Parsing JavaScript
const esprima = require("esprima");
const ast = esprima.parseScript("let x = 10;", {
range: true,
loc: true
});
console.log(ast.body[0].type); // "VariableDeclaration"
📝 Handling JSDoc Comments: Doctrine vs JSDoc
While parsers handle code, you often need to understand the comments attached to that code. This is where doctrine and jsdoc diverge in purpose.
doctrine is a low-level parser for JSDoc comments. It turns comment strings into structured objects. It does not generate documentation pages.
// doctrine: Parsing a comment string
const doctrine = require("doctrine");
const comment = "/** @param {string} name */";
const parsed = doctrine.parse(comment, { unwrap: true });
console.log(parsed.tags[0].title); // "param"
console.log(parsed.tags[0].type.name); // "string"
jsdoc is a complete documentation generator. It parses comments and code, then renders HTML pages using templates. It is heavier and intended for final output, not intermediate analysis.
// jsdoc: CLI usage via config file
// conf.json
{
"source": { "include": ["src"] },
"opts": { "destination": "./docs" }
}
// Command line
// npx jsdoc -c conf.json
🌳 AST Compliance and Standards
Interoperability matters when swapping tools. Both code parsers aim for ESTree compliance, but their history differs.
acornfollows the ESTree spec closely. It is the basis forespree, which powers ESLint. This makes it safe for tools that need to match ESLint's understanding of code.esprimaalso targets ESTree but has historically had minor deviations. It is still compatible with many tools but lacks the plugin ecosystem of Acorn for new syntax.
// acorn: Using a plugin for new syntax
const acorn = require("acorn");
const jsx = require("acorn-jsx");
const parser = acorn.Parser.extend(jsx());
const ast = parser.parse("<div />", { ecmaVersion: 2020 });
// esprima: JSX support built-in (older approach)
const esprima = require("esprima");
const ast = esprima.parseScript("<div />", {
jsx: true
});
🔌 Extensibility and Plugins
Tooling needs to adapt to new language features without waiting for core updates.
acorn uses a plugin interface. You can mix and match plugins for JSX, TypeScript, or custom syntax. This keeps the core small.
// acorn: Custom plugin usage
const acorn = require("acorn");
const bigint = require("acorn-bigint");
const parser = acorn.Parser.extend(bigint);
const ast = parser.parse("let x = 10n;");
esprima does not support plugins in the same way. Features must be added to the core library. This makes it stable but slower to adopt new standards.
// esprima: No plugin system
// Features depend on the core release version
const esprima = require("esprima");
// Cannot easily extend syntax without fork or patch
doctrine is focused solely on comment tags. It is extensible via options like recoverable to handle broken comments without crashing.
// doctrine: Error recovery
const doctrine = require("doctrine");
const badComment = "/** @param {string */";
const parsed = doctrine.parse(badComment, { recoverable: true });
// Returns partial data instead of throwing
jsdoc uses templates and plugins to modify output. You can write plugins to inject data into the generated HTML.
// jsdoc: Plugin in conf.json
// conf.json
{
"plugins": ["plugins/markdown"]
}
🛠 Real-World Scenarios
Scenario 1: Building a Linter Rule
You need to check if functions have JSDoc types.
- ✅ Best choice:
doctrine+acorn(via ESLint) - Why? You need to parse code structure and comment tags separately.
// Example logic
const codeAST = acorn.parse(code);
const commentAST = doctrine.parse(comment);
// Compare codeAST params with commentAST tags
Scenario 2: Generating API Reference
You want a website showing all public classes.
- ✅ Best choice:
jsdoc - Why? It handles templates, linking, and output generation out of the box.
# Command
npx jsdoc src/*.js -d docs
Scenario 3: Custom Code Transformation
You are building a bundler that needs to strip console logs.
- ✅ Best choice:
acorn - Why? Fast parsing and easy AST traversal.
// acorn: Traverse and modify
const ast = acorn.parse(code);
// Walk AST, find CallExpression, remove if callee is "console.log"
Scenario 4: Legacy Tool Maintenance
You are fixing bugs in a 5-year-old analysis tool.
- ✅ Best choice:
esprima - Why? Changing parsers might break existing assumptions about the AST shape.
// esprima: Stable legacy parsing
const ast = esprima.parseScript(legacyCode);
// Expect consistent output matching old logic
📊 Summary Table
| Feature | acorn | esprima | doctrine | jsdoc |
|---|---|---|---|---|
| Primary Use | Code Parsing | Code Parsing | Comment Parsing | Doc Generation |
| Output | AST (ESTree) | AST (ESTree) | Comment AST | HTML Files |
| Extensibility | ✅ Plugins | ❌ Core Only | ⚠️ Options | ✅ Templates/Plugins |
| Performance | 🚀 Very Fast | 🐢 Moderate | 🚀 Fast | 🐢 Heavy |
| Maintenance | 🟢 Active | 🟡 Stable/Legacy | 🟢 Active | 🟢 Active |
💡 The Big Picture
acorn is the modern standard for reading JavaScript code. It is fast, flexible, and built for today's tooling needs. Use it when you need to understand or change code structure.
esprima is a reliable legacy option. It works well but lacks the speed and plugin support of Acorn. Stick with it only if you are maintaining older systems.
doctrine is the specialist for comments. It gives you raw data from JSDoc tags without the overhead of generating docs. Use it inside linters or build scripts.
jsdoc is the complete solution for documentation. It takes source code and comments to produce readable websites. Use it when you need to publish API references for humans.
Final Thought: These tools often work together. A linter might use acorn to read code and doctrine to read comments. A documentation system might use jsdoc to do both. Choose based on whether you need to analyze code, analyze comments, or publish results.
How to Choose: doctrine vs acorn vs esprima vs jsdoc
- doctrine:
Choose
doctrinewhen you need to parse JSDoc comments programmatically within a linter or custom tool. It is ideal for extracting type information or tags from comments without generating full documentation. It is the standard choice for ESLint plugins that validate JSDoc. - acorn:
Choose
acornfor modern build tools, linters, or code analysis utilities where performance and standards compliance are critical. It is the default parser for many contemporary tools like Rollup and ESLint (via espree). Its plugin system allows you to support newer syntax without waiting for core updates. - esprima:
Choose
esprimaif you are maintaining legacy tooling that depends on its specific AST output or if you need a stable, unchanged parser for long-term support. However, for new projects,acornis generally preferred due to better active maintenance and speed. - jsdoc:
Choose
jsdocwhen your goal is to generate static HTML documentation sites for your API. It handles the entire workflow from parsing comments to rendering templates. It is not suitable for lightweight comment parsing within a build step wheredoctrinewould be more efficient.
Popular Comparisons
README for doctrine
Doctrine
Doctrine is a JSDoc parser that parses documentation comments from JavaScript (you need to pass in the comment, not a whole JavaScript file).
Installation
You can install Doctrine using npm:
$ npm install doctrine --save-dev
Doctrine can also be used in web browsers using Browserify.
Usage
Require doctrine inside of your JavaScript:
var doctrine = require("doctrine");
parse()
The primary method is parse(), which accepts two arguments: the JSDoc comment to parse and an optional options object. The available options are:
unwrap- set totrueto delete the leading/**, any*that begins a line, and the trailing*/from the source text. Default:false.tags- an array of tags to return. When specified, Doctrine returns only tags in this array. For example, iftagsis["param"], then only@paramtags will be returned. Default:null.recoverable- set totrueto keep parsing even when syntax errors occur. Default:false.sloppy- set totrueto allow optional parameters to be specified in brackets (@param {string} [foo]). Default:false.lineNumbers- set totrueto addlineNumberto each node, specifying the line on which the node is found in the source. Default:false.range- set totrueto addrangeto each node, specifying the start and end index of the node in the original comment. Default:false.
Here's a simple example:
var ast = doctrine.parse(
[
"/**",
" * This function comment is parsed by doctrine",
" * @param {{ok:String}} userName",
"*/"
].join('\n'), { unwrap: true });
This example returns the following AST:
{
"description": "This function comment is parsed by doctrine",
"tags": [
{
"title": "param",
"description": null,
"type": {
"type": "RecordType",
"fields": [
{
"type": "FieldType",
"key": "ok",
"value": {
"type": "NameExpression",
"name": "String"
}
}
]
},
"name": "userName"
}
]
}
See the demo page more detail.
Team
These folks keep the project moving and are resources for help:
- Nicholas C. Zakas (@nzakas) - project lead
- Yusuke Suzuki (@constellation) - reviewer
Contributing
Issues and pull requests will be triaged and responded to as quickly as possible. We operate under the ESLint Contributor Guidelines, so please be sure to read them before contributing. If you're not sure where to dig in, check out the issues.
Frequently Asked Questions
Can I pass a whole JavaScript file to Doctrine?
No. Doctrine can only parse JSDoc comments, so you'll need to pass just the JSDoc comment to Doctrine in order to work.
License
doctrine
Copyright JS Foundation and other contributors, https://js.foundation
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
esprima
some of functions is derived from esprima
Copyright (C) 2012, 2011 Ariya Hidayat (twitter: @ariyahidayat) and other contributors.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
-
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
-
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL
closure-compiler
some of extensions is derived from closure-compiler
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/
Where to ask for help?
Join our Chatroom