Pre-commit hooks

A hook runs jscpd on the developer's machine at git commit, before the change reaches a branch anyone else sees. It is the earliest gate and the cheapest one: no runner, no checkout, a scan that takes milliseconds, and the commit goes through or it does not. This page sets the hook up with the pre-commit framework, with Husky and with a plain git hook, then shows how to keep it from blocking on duplication that was already there. The shared gate for pull requests is on the CI page.

Minimal setup

pre-commit framework

repos:
  - repo: https://github.com/kucherenko/jscpd
    rev: v5.4.0
    hooks:
      - id: jscpd

The hook is defined in the jscpd repository: on every commit it runs jscpd --exit-code 1 on the staged text files and blocks the commit when two of them hold a clone. Since jscpd 5.2.1 it is a language: python hook that installs the PyPI wheel of the pinned release into pre-commit's own environment, so the machine needs no Node.js; pre-commit autoupdate moves rev to the latest tag.

pre-commit install

pre-commit run jscpd --files src/orders.js src/reports/orders.js

Check for duplicated code................................................Failed
- hook id: jscpd
- exit code: 1

Clone found (javascript)
 - orders.js [1:1 - 12:2] (12 lines, 124 tokens)
   reports/orders.js [1:1 - 12:2]
Found 1 clones.

The two files are the exact pair of fixtures/mcp-demo; a staged file whose twin is not staged passes, because the hook sees the staged files only.

Husky

npm install -D husky jscpd

npx husky init

npx jscpd . --threshold 5

husky init creates .husky/pre-commit with npm test in it and adds a prepare script, so every npm install wires the hook; replace the file's content with the jscpd command. With jscpd in devDependencies the hook runs the version from the lockfile; npx jscpd@5 works without the install and downloads the package on its first run.

A plain git hook

#!/bin/sh
jscpd . --threshold 5

chmod +x .githooks/pre-commit

git config core.hooksPath .githooks

Hooks under .git/hooks are not versioned, so the script lives in the repository and git is pointed at the directory; each clone of the repository runs the git config command once, from a prepare script in package.json, a make hooks target or the onboarding notes. This hook runs whatever jscpd is on the PATH: npm install -g jscpd@5, pip install jscpd or the installer from the installation page.

Make it useful

Scan the whole tree and let legacy clones through

repos:
  - repo: https://github.com/kucherenko/jscpd
    rev: v5.4.0
    hooks:
      - id: jscpd
        pass_filenames: false
        always_run: true
        args: ["src", "--baseline", ".jscpd-baseline.json", "--fail-on-new-clones"]

pass_filenames: false stops pre-commit from handing the staged files to jscpd and always_run: true runs the hook even when no staged file is code, so jscpd scans src as a whole and sees a clone between a staged file and one nobody touched. The baseline lets the clones recorded in .jscpd-baseline.json through and fails the commit on a new one, with a [NEW] marker on the pair; args replaces the hook's own --exit-code 1, so the gate is what you put there. The baseline guide shows how to record and refresh the file.

Gate on a percentage

      - id: jscpd
        require_serial: true
        args: ["--threshold", "5"]

With args set this way, the staged files still follow at the end of the command line, and the hook fails when their duplication is above 5 percent (strictly greater), which lets a small clone among large files through. require_serial: true keeps all staged files in one jscpd process; without it pre-commit splits them over several processes, and a pair can be split with them.

Scope the hook

      - id: jscpd
        files: \.(ts|tsx)$
        args: ["--exit-code", "1", "--ignore", "**/__snapshots__/**"]

pre-commit's files and exclude patterns decide which staged files reach the hook (it declares types: [text], so every text file does by default), and jscpd's --format and --ignore filter again inside the run; a file outside the --format list is left out even when pre-commit passes it. The format names are on the supported formats page.

Reference

• CLI options for every flag the args list can take.
• Exit codes for what makes the hook fail.
• Configuration file, since a .jscpd.json in the repository root applies to the hook too: git runs hooks from the root of the working tree.

Troubleshooting

The hook passes although two staged files are clones

pre-commit spreads the staged files over several jscpd processes, one per CPU core, so the two files of a pair can land in different batches and no process sees both; the output then holds two statistics tables. Set require_serial: true on the hook to keep the files in one process, or scan the tree with pass_filenames: false as above.

error: invalid value 'src/orders.js' for '--exit-code [<EXIT_CODE>]'

--exit-code and --fail-on-new-clones take an optional number, so a file name right after them is read as that number and jscpd exits 2. Always give the number in a hook, --exit-code 1 or --fail-on-new-clones 0, because pre-commit appends the file names at the end of the command line.

The commit is blocked by duplication that was already there

--exit-code 1 fails on any clone among the files it sees, old or new. Record the accepted state in a baseline and gate on new clones only, as above, or switch the args to --threshold.

The table lists bash and perl files the project does not have

A scan of the repository root (.) walks .git as well, and the sample hooks in .git/hooks count as shell and Perl sources. Pass the source directories (src) instead of ., or add --ignore "**/.git/**".

Node.js errors with an older rev

Up to v5.2.0 the hook was language: node and needed Node.js; from v5.2.1 it installs the PyPI wheel. Move rev forward. The Husky and plain hooks run whatever jscpd the PATH resolves, so install it where the hook runs.

Skipping the hook once

SKIP=jscpd git commit skips this hook in the pre-commit framework; git commit --no-verify skips every hook of any kind.

Related

• Use jscpd in CI for the shared gate that a skipped hook cannot bypass.
• Fail only on new duplication for the baseline the whole-tree hook relies on.
• Exit codes for the codes a hook turns into a blocked commit.
• Installation for every way to put jscpd on the PATH.