Monorepos

In a repository of many packages a clone means different things depending on where its two halves live. Inside one package it is a refactoring for that team; across two packages it is a candidate for a shared library; between packages owned by different teams it may be a copy both sides made on purpose. One scan reports all of them, and the question this page answers is how to keep the ones you will act on. The examples run on a small layout with two kinds of copy:

Run it

jscpd .

Clone found (javascript)
 - packages/billing/src/invoice.js [1:1 - 12:20] (12 lines, 83 tokens)
   packages/billing/src/receipt.js [1:1 - 12:20]
Clone found (javascript)
 - packages/billing/src/invoice.js [1:1 - 12:20] (12 lines, 83 tokens)
   packages/shipping/src/labels.js [1:1 - 12:20]
Clone found (javascript)
 - packages/shipping/src/rates.js [1:1 - 9:13] (9 lines, 88 tokens)
   shared/src/money.js [1:1 - 9:13]
Found 3 clones.

All three pairs, as a plain scan should. --skip-local keeps the clones that cross from one scan path to another and drops the pairs inside one path, so each package goes on the command line as a path of its own:

jscpd packages/billing packages/shipping shared --skip-local --reporters ai

Clones:
src/ invoice.js:1-12 ~ labels.js:1-12
src/ rates.js:1-9 ~ money.js:1-9
---
2 clones · 38.9% duplication

The pair inside packages/billing is gone. --skip-isolated draws the line between named folders instead: clones between members of a group are skipped, and clones inside a member or against anything outside the group stay.

jscpd . --skip-isolated "packages/billing|packages/shipping" --reporters ai

Clones:
packages/billing/src/ invoice.js:1-12 ~ receipt.js:1-12
packages/shipping/src/rates.js:1-9 ~ shared/src/money.js:1-9
---
2 clones · 38.9% duplication

Now the pair between billing and shipping is gone, the copy inside billing is back, and the copy from the shared library is reported in both runs, because shared belongs to no group.

Reading the output

The ai reporter prints the common prefix of a pair once and then the two fragments, packages/ billing/src/invoice.js:1-12 ~ shipping/src/labels.js:1-12, which is why it suits a log with many packages. With several scan paths, jscpd reports each file relative to its own scan path, which is where the src/invoice.js and src/labels.js of the second run come from; --absolute prints full paths when that is ambiguous. The duplication percentage counts the reported clones only, 61.1 % in the first run and 38.9 % in the other two, so a --threshold measures what is left after the skips.

Common tasks

Which package holds the duplication

jscpd . --summary --summary-top 2 --reporters ai

Clones:
packages/billing/src/ invoice.js:1-12 ~ receipt.js:1-12
packages/ billing/src/invoice.js:1-12 ~ shipping/src/labels.js:1-12
packages/shipping/src/rates.js:1-9 ~ shared/src/money.js:1-9
---
3 clones · 61.1% duplication
---
Summary by tokens (5 files, 3 folders):
files (tokens/lines/size/cx/dup%):
packages/shipping/src/rates.js 92/9/342/3/100.0%
shared/src/money.js 92/9/344/3/100.0%
folders (files/tokens/lines/size):
packages/shipping/src 2/177/21/664
packages/billing/src 2/170/24/648

--summary appends the largest files and folders with their size, complexity and, for files, the share of lines inside clones; --summary-by lines, size or complexity changes the ranking, and the console reporter prints the same lists as tables. Folders aggregate their direct children, so the rows are the leaf directories; a large file with a high dup% is the first refactoring target.

One configuration for the whole repository

{
  "ignore": ["**/node_modules/**", "**/dist/**"],
  "skipIsolated": [["packages/billing", "packages/shipping"]],
  "reporters": ["console", "sarif"]
}

jscpd reads .jscpd.json from the directory you run it in, or the file --config names, so a root config applies to jscpd . from the root and to a run from inside a package only with --config ../../.jscpd.json. skipIsolated takes nested arrays, one per group, and "skipLocal": true is the other flag. Inside a git checkout jscpd already skips what .gitignore lists; the ignore patterns matter outside git or with --no-gitignore, where the node_modules of every package would be scanned.

In CI

- uses: kucherenko/jscpd@v5
  with:
    path: packages/billing packages/shipping shared
    skip-local: true
    config: .jscpd.json
    threshold: 3

path takes several space-separated paths, so this is the second run above as a check, with the isolation groups coming from the config file: the Action has no skip-isolated input, so the groups go through config or through extra-args: --skip-isolated "packages/billing|packages/shipping". For a gate per package, a matrix over the package names with path: packages/${{ matrix.package }} gives each one its own threshold and its own check in the pull request. The GitHub Action reference lists the inputs.

Limits

• --skip-local needs two or more paths; with one path every clone lies inside it, and jscpd . --skip-local reports nothing.
• --skip-isolated matches folders by path prefix, written as you would type them on the command line, relative to the current directory: from inside packages/, the group is billing|shipping.
• Skipped clones leave the statistics too, so the percentage, --threshold and the baseline see only what stays.
• --format, --ignore and the thresholds apply to the whole run. When packages need different settings, run jscpd once per package with its own --config.

Options

CLI options has every flag with its default; the keys go in .jscpd.json.

Related

• Use jscpd in CI for the workflow around the per-package check.
• Fail only on new duplication when the packages carry duplication you will not remove now.
• Cross-format detection when a JavaScript package and a TypeScript package share code.
• Configuration for the keys to set first in a root .jscpd.json.