Cross-format detection

By default jscpd compares a file only with files of the same format, so a function copied from a .js file into a .ts file and given type annotations is invisible to a plain scan. Two things change that. --cross-formats puts related formats into one comparison pool, with the types stripped from the TypeScript side, and files that embed other languages, Vue, Svelte and Astro components and Markdown, are split into blocks that each join the pool of their own language. You reach for the first during a migration from JavaScript to TypeScript or in a repository that keeps both; the second needs no flag.

Run it

fixtures/cross-formats holds a.ts, a typed computeTotals under an interface, and b.js, the same function without types. A plain scan keeps the two formats apart:

jscpd .

No duplicates found.
Found 0 clones.

jscpd . --cross-formats js-ts

Clone found (typescript)
 - a.ts [6:1 - 19:2] (14 lines, 71 tokens)
   b.js [1:1 - 14:2]
Found 1 clones.

js-ts is the preset for javascript,jsx,typescript,tsx; --cross-formats "javascript,typescript" names the same two formats by hand.

Reading the output

The clone is reported on the original sources: lines 6 to 19 of a.ts, annotations included, against lines 1 to 14 of b.js. The match itself runs on the TypeScript tokens with the type syntax removed, which is how function computeTotals(values: number[], threshold: number): Totals lines up with function computeTotals(values, threshold) and the as Totals at the end disappears; the interface above the function has no counterpart, so the clone starts below it. In the table a cross-format clone is attributed to one format of the group, here typescript, while the Total row counts it once.

Common tasks

Group other formats

jscpd . --cross-formats "javascript,typescript;css,scss"

Formats inside a group are separated by commas and groups by semicolons. Type stripping happens only in a group that holds JavaScript and TypeScript; any other group compares the token streams as they are, so a rule written the same way in a .css and a .scss file matches and a nested SCSS rule does not. A group with one format is dropped with a warning, Warning: --cross-formats group 'javascript' has fewer than two formats, ignored. In the config file the key is crossFormats:

{
  "crossFormats": [["javascript", "typescript"], ["css", "scss"]]
}

The key also takes the command-line string, "crossFormats": "js-ts", or an array of such strings, ["javascript,typescript", "css,scss"]. A flat array of single names, ["javascript", "typescript"], is read as one group per name and ignored with the warning above.

Clones inside components and Markdown

jscpd splits a .vue, .svelte or .astro file into its blocks and a Markdown file into its fenced code blocks, and scans each block under the block's own language: a <script lang="ts"> as typescript, a <style> as css (or scss and less), the markup as html, the frontmatter of an Astro file as typescript, a ```ts fence as typescript. Markdown prose is scanned as markdown (5.0.14+). A block takes part in the pool of its format, so a <script> matches a .ts file and a fence in a guide matches the component it documents. The report names the block's format after the file. In fixtures/embedded-stats-demo, two Vue components and two Markdown guides share 13 lines of TypeScript:

jscpd .

Clone found (typescript)
 - component/ExportCard.vue:typescript [35:1 - 48:2] (13 lines, 189 tokens)
   component/ReportCard.vue:typescript [35:1 - 48:2]
Clone found (typescript)
 - component/ExportCard.vue:typescript [35:1 - 48:2] (13 lines, 189 tokens)
   markdown/exporting.md:typescript [23:1 - 57:2]
Clone found (typescript)
 - component/ExportCard.vue:typescript [35:1 - 48:2] (13 lines, 189 tokens)
   markdown/reporting.md:typescript [23:1 - 57:2]
Found 3 clones.

The Markdown fragment runs from line 23 to line 57 because the shared code sits in two fences with prose between them; the console counts the 13 lines the fences duplicate, and the prose stays in the markdown row. The typescript row holds only the lines of the blocks, 52 over four sources, since jscpd 5.3.2; the vue and markdown rows hold the rest of each file. The pattern is the same for the other formats: a Svelte component yields component1.svelte:javascript, :css and :html, an Astro component component1.astro:typescript for the frontmatter and :html for the markup.

In CI

- uses: kucherenko/jscpd@v5
  with:
    extra-args: --cross-formats js-ts
    threshold: 3

The Action has no cross-formats input, so the flag goes through extra-args or through the crossFormats key of a config file passed with config; embedded blocks need nothing. A cross-format clone counts in the duplication percentage like any other, so the threshold sees it, and the SARIF upload links both files in the alert.

Limits

• --format selects files by the file's format, not by the block's. --format typescript leaves .vue and .md files out, so a TypeScript gate that should see components lists typescript,vue, and markdown for the guides.
• Only type syntax is removed from the TypeScript side. Constructs that compile to code, such as an enum, have no JavaScript twin to match.
• Groups compare token streams, so formats whose syntax differs beyond types, SCSS nesting against flat CSS for instance, only match where the text is the same.
• In the per-format table a cross-format clone belongs to one of the group's formats, so the rows of the others show fewer clones than the files take part in.

Options

CLI options has every flag with its default; the format names are on the supported formats page.

Related

• Supported formats for the names a group takes and the extensions behind them.
• How detection works for the tokenization and normalization the pool is built on.
• Monorepos for a JavaScript package and a TypeScript package that share code.
• Types of code clones for the renamed and near-miss clones a cross-format pool can also report.