Fail only on new duplication

Most codebases carry duplication that nobody is going to remove this sprint, and a percentage threshold handles it badly: set high, it lets a fresh copy-paste through under the limit; set low, it trips on legacy code the change never touched. A baseline separates the two. The clones that exist today are recorded and accepted, and a build fails only when a change adds a clone that is not among them. You reach for it when you want a gate on pull requests from day one, on a codebase that is not clean.

Run it

The commands run on a copy of fixtures/mcp-demo/src, which holds three exact clones. The first run records them:

jscpd src --baseline .jscpd-baseline.json --update-baseline --reporters ai

Baseline .jscpd-baseline.json updated: 3 fingerprints added, 0 removed (3 total)
Clones:
invoice.js:1-6 ~ print/invoice.js:1-6
invoice.js:6-12 ~ print/invoice.js:7-13
orders.js:1-12 ~ reports/orders.js:1-12
---
3 clones · 36.2% duplication

{
  "version": 1,
  "fingerprints": {
    "0f6fc236d5d07e6b": 1,
    "de96aa427b11c6eb": 1,
    "eac17f2cc970be19": 1
  }
}

Commit the file. From now on the gate is one flag:

jscpd src --baseline .jscpd-baseline.json --fail-on-new-clones

It prints the three clones without a marker and exits 0. Copy returns.js to refunds.js and run the same command again:

Clone found (javascript)
 - invoice.js [1:1 - 6:72] (6 lines, 105 tokens)
   print/invoice.js [1:1 - 6:72]
Clone found (javascript)
 - invoice.js [6:71 - 12:2] (7 lines, 93 tokens)
   print/invoice.js [7:53 - 13:2]
Clone found (javascript)
 - orders.js [1:1 - 12:2] (12 lines, 124 tokens)
   reports/orders.js [1:1 - 12:2]
Clone found (javascript) [NEW]
 - refunds.js [1:1 - 10:2] (10 lines, 104 tokens)
   returns.js [1:1 - 10:2]
Found 4 clones (1 new).
ERROR: jscpd found 1 new clones not in the baseline (allowed: 0)

The run exits 1. The three old clones are still listed and still count in the percentage; only the fourth fails the build.

Reading the output

Each key in the baseline file is a hash of a duplicated fragment's content, and the value is how many clones share it. A clone is new when its hash is absent, or when more clones carry it than the file allows, so a third copy of a fragment that has one recorded twin is new. Nothing about paths or line numbers goes into the hash: moving a file, shifting lines above the fragment or switching line endings leaves the gate quiet, and the file diffs one line per fingerprint when you refresh it.

The console reporters mark a new clone with [NEW] and count them in Found 4 clones (1 new). The other reporters carry the same information: json adds isNew to each clone and newClones and newDuplicatedLines to the totals, sarif reports a new clone at level error and the rest at warning, so GitHub code scanning highlights only the regression, codeclimate gives it severity major against minor, and openmetrics adds the jscpd_new_clones and jscpd_new_duplicated_lines gauges.

The percentage in the table counts every clone, old and new, so a --threshold in the same run keeps watching the whole picture; when either gate trips, the run exits 1.

Common tasks

Accept new duplication in a pull request

jscpd src --baseline .jscpd-baseline.json --update-baseline --reporters silent

Baseline .jscpd-baseline.json updated: 1 fingerprints added, 0 removed (4 total)
Duplications detection: Found 4 exact clones with 35(44.30%) duplicated lines in 7 (1 formats) files.

Commit the file with the change; the diff shows one added fingerprint, so the reviewer sees that the pull request accepts one more clone. The same command after a refactoring drops what is gone. After deleting refunds.js and reports/orders.js:

Baseline .jscpd-baseline.json updated: 0 fingerprints added, 2 removed (2 total)
Duplications detection: Found 2 exact clones with 13(22.81%) duplicated lines in 5 (1 formats) files.

Allow a budget of new clones

jscpd src --baseline .jscpd-baseline.json --fail-on-new-clones 1 --reporters ai

Clones:
invoice.js:1-6 ~ print/invoice.js:1-6
invoice.js:6-12 ~ print/invoice.js:7-13
orders.js:1-12 ~ reports/orders.js:1-12
refunds.js:1-10 ~ returns.js:1-10
---
4 clones · 44.3% duplication

The run exits 0: one new clone is within the allowance, a second one would fail. Without a number the allowance is 0.

Baseline from a git ref

When you would rather keep nothing in the repository, --baseline-from-ref builds the baseline from a git ref's tree: jscpd checks the ref out into a temporary worktree, scans it with the same configuration and compares the two results in memory. On a branch that adds refunds.js on top of main:

jscpd src --baseline-from-ref main --fail-on-new-clones --reporters ai

Clones:
invoice.js:1-6 ~ print/invoice.js:1-6
invoice.js:6-12 ~ print/invoice.js:7-13
orders.js:1-12 ~ reports/orders.js:1-12
refunds.js:1-10 ~ returns.js:1-10
---
4 clones · 44.3% duplication
ERROR: jscpd found 1 new clones not in the baseline (allowed: 0)

The run scans twice, the base tree and the working tree, and the ref has to exist locally. In a shallow CI checkout origin/main is missing, and jscpd exits 1 with git ref 'origin/main' not found and the advice to fetch it; git fetch origin main followed by --baseline-from-ref FETCH_HEAD is the smallest fix, a full fetch the usual one. The flag cannot be combined with --baseline.

In CI

on: [pull_request]

jobs:
  jscpd:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: kucherenko/jscpd@v5
        with:
          baseline: .jscpd-baseline.json
          fail-on-new-clones: true
          reporters: console,sarif

fail-on-new-clones: true is the bare flag, an integer is the allowance. For the ref variant, replace baseline with baseline-from-ref: origin/${{ github.base_ref }} and give actions/checkout fetch-depth: 0. On GitLab, the script line is jscpd . --baseline .jscpd-baseline.json --fail-on-new-clones --reporters console,codeclimate,openmetrics, and the CI guide has the artifacts block around it, so the merge request shows the new clone as a major Code Quality issue and the metrics widget shows the jscpd_new_clones delta.

Limits

• Put the paths before --fail-on-new-clones or give it a number. The flag takes an optional value, so --fail-on-new-clones src reads src as the number and exits 2 with error: invalid value 'src' for '--fail-on-new-clones [<N>]'. The GitHub Action orders the arguments itself.
• --update-baseline needs --baseline and exits 1 without it; a --baseline that names a missing file exits 1 and tells you to run with --update-baseline once.
• Record and gate with the same options. The file lists what one run found, so a run with --ignore-identifiers or --max-gap-lines added finds clones the file does not know, and they count as new.
• --dashboard and --health do not build a baseline: they warn and ignore --baseline, --baseline-from-ref and --update-baseline, and they refuse --fail-on-new-clones with an error. --complexity never detects clones, so it ignores the whole family with a warning. See the dashboard guide.
• --baseline-from-ref runs git, so it does not work in the ghcr.io/kucherenko/jscpd image, which holds only the binary.

Options

CLI options lists them with the rest of the flags; the keys go in .jscpd.json.

Related

• Use jscpd in CI for the workflows around the gate on GitHub and GitLab.
• Pre-commit hooks for the same baseline in a hook, so legacy clones do not block commits.
• Duplication over git history for the trend of the number the threshold watches.
• Exit codes for every code a gated run can return.