firecrawl/pdf-inspector

Fast Rust library for PDF inspection, classification, and text extraction. Intelligently detects scanned vs text-based PDFs to enable smart routing decisions.

Rust★ 18,078+541 that day#7 on trending

Analyzed September 2, 2026 at 65b7fa1 (main) with jscpd 5.4.0 in 1.1s. Code duplication went from 6.18% on Sep 1 to 6.17% on Sep 2.

6.17%duplicated code · 6,313 of 102,240 lines

514clones in code

70code files of 126 scanned

6.63%duplicated over all files · 587 clones

The code number leaves out tests, fixtures, docs, examples, data files, styles, translations and vendored code.

Largest clones in the code

17 lines in 4 placesrust148 tokens

src/tables/detect_lines.rs:2425–2441src/tables/detect_lines.rs:2478–2494src/tables/detect_lines.rs:2504–2520src/tables/detect_lines.rs:2531–2547

2425    make_hline(440.0, 80.0, 420.0, 1),
2426];
2427let items = vec![
2428    make_item("Model", 100.0, 490.0, 1),
2429    make_item("Accuracy", 220.0, 490.0, 1),
2430    make_item("Latency", 340.0, 490.0, 1),
2431    make_item("Alpha", 100.0, 465.0, 1),
2432    make_item("91.2", 220.0, 465.0, 1),
2433    make_item("12", 340.0, 465.0, 1),
2434    make_item("Beta", 100.0, 450.0, 1),

First 10 of 17 lines.

33 lines in 3 placesrust196 tokens

src/detector.rs:3183–3215src/detector.rs:3426–3458src/detector.rs:3683–3715

3183let content_id = doc.add_object(Object::Stream(lopdf::Stream::new(
3184    dictionary! {},
3185    content_data.to_vec(),
3186)));
3187let resources = dictionary! {
3188    "Font" => dictionary! {
3189        "F1" => Object::Reference(bad_font_id),
3190    },
3191    "XObject" => dictionary! {
3192        "XF1" => Object::Reference(xobj_id),

First 10 of 33 lines.

16 lines in 4 placesrust88 tokens

src/markdown/convert.rs:1729–1744src/markdown/convert.rs:1758–1773src/markdown/convert.rs:1837–1852src/markdown/convert.rs:1945–1960

1729page_roles.insert(1i64, StructRole::P);
1730let mut roles = HashMap::new();
1731roles.insert(1u32, page_roles);
1732
1733let md = to_markdown_from_lines_with_tables_and_images(
1734    lines,
1735    MarkdownOptions::default(),
1736    HashMap::new(),
1737    HashMap::new(),
1738    &HashMap::new(),

First 10 of 16 lines.

7 more

6 lines in 5 placesrust66 tokens
src/detector.rs:2594–2599 src/detector.rs:2650–2655 src/detector.rs:2903–2908 src/detector.rs:3152–3157 src/detector.rs:3242–3247
24 lines in 3 placesrust129 tokens
src/detector.rs:2236–2259 src/detector.rs:2278–2301 src/detector.rs:2333–2356
20 lines in 4 placesrust81 tokens
src/markdown/furniture.rs:951–970 src/markdown/furniture.rs:1020–1039 src/markdown/furniture.rs:1056–1075 src/markdown/furniture.rs:1088–1107
13 lines in 5 placesrust58 tokens
src/extractor/mod.rs:1425–1437 src/extractor/mod.rs:1684–1696 src/extractor/mod.rs:1718–1730 src/extractor/mod.rs:2567–2579 src/extractor/mod.rs:2905–2917
32 lines in 2 placesrust228 tokens
src/extractor/content_stream.rs:1206–1237 src/extractor/content_stream.rs:1240–1271
21 lines in 2 placesrust228 tokens
src/markdown/mod.rs:1088–1108 src/tables/detect_rects.rs:891–911
13 lines in 3 placesrust113 tokens
src/tables/detect_lines.rs:2622–2634 src/tables/detect_lines.rs:2655–2667 src/tables/detect_lines.rs:2698–2710

Largest clones over all files, tests and data included

80 lines in 2 placesmarkdowndocs and examples625 tokens

AGENTS.md:1–80CLAUDE.md:1–80

1# pdf-inspector
2
3Fast PDF text extraction to structured Markdown. CLI binary: `pdf2md`. Detection binary: `detect-pdf`.
4
5## Build & Test
6
7```bash
8cargo fmt                                    # format
9cargo clippy -- -D warnings                  # lint (enforced, zero warnings)
10cargo test                                   # unit + integration tests (267+ unit, 73+ integration)

First 10 of 80 lines.

17 lines in 4 placesrust148 tokens

src/tables/detect_lines.rs:2425–2441src/tables/detect_lines.rs:2478–2494src/tables/detect_lines.rs:2504–2520src/tables/detect_lines.rs:2531–2547

2425    make_hline(440.0, 80.0, 420.0, 1),
2426];
2427let items = vec![
2428    make_item("Model", 100.0, 490.0, 1),
2429    make_item("Accuracy", 220.0, 490.0, 1),
2430    make_item("Latency", 340.0, 490.0, 1),
2431    make_item("Alpha", 100.0, 465.0, 1),
2432    make_item("91.2", 220.0, 465.0, 1),
2433    make_item("12", 340.0, 465.0, 1),
2434    make_item("Beta", 100.0, 450.0, 1),

First 10 of 17 lines.

33 lines in 3 placesrust196 tokens

src/detector.rs:3183–3215src/detector.rs:3426–3458src/detector.rs:3683–3715

3183let content_id = doc.add_object(Object::Stream(lopdf::Stream::new(
3184    dictionary! {},
3185    content_data.to_vec(),
3186)));
3187let resources = dictionary! {
3188    "Font" => dictionary! {
3189        "F1" => Object::Reference(bad_font_id),
3190    },
3191    "XObject" => dictionary! {
3192        "XF1" => Object::Reference(xobj_id),

First 10 of 33 lines.

23 lines in 2 placesmarkdowndocs and examples297 tokens
docs/python.md:14–36 napi/README.md:14–38
17 lines in 2 placesmarkdowndocs and examples289 tokens
docs/python.md:14–30 docs/rust-api.md:13–29

By format, code only

Format	Files	Lines	Clones	Duplicated lines	Duplication
rust	53	99,722	513	6,301	6.32%
python	4	1,265	1	12	0.95%
bash	9	644	0	0	0%
javascript	3	435	0	0	0%
perl	1	174	0	0	0%

By format, all files

Format	Files	Lines	Clones	Duplicated lines	Duplication
rust	57	104,624	569	7,243	6.92%
yaml	6	1,108	7	86	7.76%
markdown	19	2,577	5	134	5.2%
python	10	2,439	3	33	1.35%
markup	2	1,184	1	9	0.76%
bash	16	724	1	7	0.97%
text	3	91	1	24	26.37%
javascript	4	450	0	0	0%
toml	5	207	0	0	0%
perl	1	174	0	0	0%

Trending appearances

Day	Rank	Stars	Files	Lines	Clones	All files	Code	Health	Commit
Sep 2, 2026	#7	18,078 (+541)	126	113,725	587	6.63%	6.17%	—	`65b7fa1`
Sep 1, 2026	#8	17,674 (+228)	126	113,659	587	6.63%	6.18%	—	`23cf1ad`

Want this for your own project? Install jscpd and run jscpd ., or measure it the way the baselines were measured.