Dead-code removal on four open-source Java projects: the CodeLaser engine against Claude Code

In short

What the experiments show

On four open-source Java projects, the CodeLaser engine removed dead code 8 to 14 times faster than Claude Code, and every result was clean. Claude Code’s results compiled, but in five of its six attempts they broke a build, a test suite or a public API.

• The engine removed everything unreachable in 1 to 7 minutes per project. Every project still builds, the checks found no fault, the test suites were left intact, and no test that was run regressed.
• The agent took 8 to 14 times as long and used 5.2 to 15.6 million input tokens per attempt.
• The agent’s extra removals broke things the compiler does not check. It disabled every HTTP write in QuestDB: 133 test classes fail. Timefold Solver’s own build rejected both attempts, so 855 of its 6,269 tests never ran. On Trino, 885 public classes can now be created from outside. The engine did none of this.
• The agent’s behaviour is non-deterministic. As an example, two attempts on Caffeine with the same input removed 6 and 21 declarations and agreed on only 5.

Every run is below, on one time axis. Green is a clean result, amber a problem the checks found, red a broken build or failing tests.

Every result in this document was measured by a predefined test harness, independently of both sides. It parsed each project’s source before and after every run, with the same parser for both sides, and compared the two. Neither the engine’s nor the agent’s own report of what it had done was used.

Removing dead code is an important and non-trivial task in fighting technical debt. The task is complex and sometimes needs contextual judgement, so having an agent in the loop is a good idea. The finding is about the division of labour: the agent should drive a tool that works out what is dead, not write that tool itself. Handed this task, an agent starts by building a dead-code analysis of its own. All six attempts here did exactly that, unprompted. The analysis has to resolve every name in a Java program and know every way a declaration stays alive without being referenced. That is a very complex program. An agent cannot be expected to write it on the spot, in answer to a refactoring task such as dead-code removal.

In this benchmark the engine ran on its own, with no agent and no tokens. It can also be handed to an agent. The division of work then becomes more natural. The engine does the code mechanics: resolving every name, working out what nothing reaches, and making the removal. The agent decides what to tackle, judges the findings and explains the result. Its tokens then go on reasoning and on talking to you.

The setup

What was compared

Dead code is code that nothing can reach: a method nobody calls, a class nobody names, a field nobody reads. Removing it is useful, tedious and risky, because "nothing reaches this" is a claim about the whole program.

Two approaches were compared on the same task. The CodeLaser engine runs its dead-code operation on a fully resolved model of the project: it works out what refers to what, and deletes what cannot be reached. Claude Code is the coding agent a developer would use interactively, here working on its own with full access to the project and a shell. It was Claude Code 2.1.283 on Claude Opus 5.5 in all six attempts. Where this document says the agent, it means Claude Code.

Claude Code was used before the runs to propose the list of roots, described below, that both approaches were given as input. The engine itself ran on its own, without Claude.

Both sides were run and measured by the same harness, which never asks either side what it did. It counts what each run removed, runs checks over the result, and then runs the project’s own tests. Appendix A describes it.

Caffeine is a high-performance in-memory caching library, small and mature. Timefold Solver is an optimisation engine for problems such as vehicle routing and shift rostering, with a heavy test suite. Trino is a distributed SQL query engine, spread across 110 separately built Maven modules. QuestDB is a time-series database. Each project was pinned to one commit and reset before every run.

The CodeLaser engine

How the CodeLaser engine removes dead code

Finding dead code has a cheap approximation, while fully removing it requires a high level of precision. Counting references and walking outward from the roots will find most of the obvious dead code, which is why all six agent attempts wrote a program to do exactly that. Deleting declarations is where things get more difficult: every place that referred to one has to be repaired, methods that only it called are now dead too, and imports, overrides and signatures have to be corrected. Each deletion can make another declaration dead, so the work repeats until a pass finds nothing.

Accurate detection based on a fully resolved model. Before it decides anything, the engine builds a model of the whole project in which every name is resolved to what it denotes: which declaration a call reaches, which method overrides which, which type a name means. The model is built primarily from the source code, and uses compiled code mainly for references into libraries. All of the engine’s reasoning happens on this model, never on bytecode. Reachability is computed over it, outward from the roots. This is what separates it from counting references. The first example under Errors in the agent’s output is a case where the difference disables a database’s write path.

The engine knows where reachability is decided outside the code. A name in a configuration file, a class loaded from a string, an enum constant selected by text at run time: none of these is a reference the compiler can see. The engine carries a rule for each such mechanism. One of them: a call to valueOf or values() on an enum marks every constant in that enum as used, because which one will be asked for is not decided until the program runs.

The engine declines what it cannot show is safe. Where it cannot show that a removal is safe, it does not make it, and says why. On QuestDB, for example, it removed eleven plain constants from one class and kept the field beside them whose initialiser registers a logger, because deleting that field would change what the program does.

The engine does not use the compiler as a source of feedback. It decides what to remove without building the project to test its guesses. The build afterwards only verifies the result.

The agent

Every agent attempt built its own dead-code analyser

This section is reconstructed from the six session transcripts and the programs each attempt left behind. All six took the same route, unprompted. Each wrote a reachability analysis of its own, from scratch, and ran it in a loop:

1. Build the project, then read the compiled class files. Three attempts used the ASM library, three the JDK’s own class-file API. In a compiled class every call already names what it reaches, so this is the quickest place to start.
2. Walk outward from the roots. Each matched the list of roots to the compiled code, followed calls, field reads and object creation, and reported whatever nothing reached.
3. Cut it out of the source. Four attempts wrote code that uses a Java parser to find each declaration in the source and delete it. The two Caffeine attempts used short Python scripts.
4. Rebuild. On failure, change a rule. When a removal broke the build, no attempt repaired the calling code. It went back to its last good state, changed a rule in its analyser, and ran again.
5. Stop when a pass finds nothing. All six said they stopped because they had run out of dead code, not out of time.

Most of the time went on discovering aspects of the code that are not available from class files alone. For example, the compiler copies constants into the classes that use them, drops code behind if (debug), and keeps some annotations only in the source. The attempts ran into these gaps one at a time and added a rule for each. None handled names in strings or configuration files; for those, all six relied on the list of roots. The engine’s model is built primarily from the source, so what the compiler copies away or leaves out of the class files is still in it. All its reasoning happens on that model, never on bytecode.

Deleting the constructors was a decision the agent made. All six attempts deleted private constructors of helper classes. The first Caffeine attempt put them back, because the project’s build tooling warned about it. The other five kept the deletions. Three of them, both Timefold Solver attempts and Trino, described the consequence only in their closing message: the classes can now be created from outside. They called it "a judgement call" and "easy to revert", and left it in place.

The real task was building a dead-code engine under a clock. The instruction was to remove dead code. What filled the time was writing a program to decide what dead code is, and every attempt wrote its own, from scratch. The analysers ran from 476 to 1,747 lines; Appendix C lists them. Reading class files was a reasonable choice, because the alternative is a Java parser that resolves every name, which is more than an afternoon’s work. But it meant rediscovering, one rule at a time, what ordinary Java already implies: an override of a default method, a private constructor, an enum constant selected by name at run time, a field whose initialiser does work. The attempts reached some of these rules and missed others. The engine carries them all. An agent that calls the engine does not have to rediscover them.

Results

What each run removed and what it cost

This section compares the two sides project by project: how long each run took, what it spent, and how its result held up. The chart in the summary shows every run at a glance. The full figures for every run are in Appendix C.

Results

Where the two sides disagree

Both sides were measured against the same parse of the same code, so their removals can be compared one declaration at a time. Nearly everything the engine removed, the agent removed too. The agent also removed a great deal more. Those extra removals are where the damage is:

On Trino, 983 of the 1,251 declarations the agent removed and the engine did not are constructors. Removing them changes the public API of the classes that lose them, which is the third example below. Appendix C breaks the extra removals down by kind.

Observation

The agent costs far more time and tokens

Every time above is the tool’s own work. The clock starts when the project’s first build starts and stops when the tool finishes. Checking the result afterwards, by compiling it again, counting it, and running the checks and the test suite, is excluded on both sides.

Most of the engine’s 6m 32s on Trino is not analysis. Working out what nothing reaches, across eleven thousand files, takes 13.1 seconds:

Deciding what is dead is the cheap part. Resolving every name in the program and then making the edits is where the time goes. An agent has to do both as well, and starts with nothing resolved.

Tokens show the same pattern. A single attempt consumed between 5.2 and 15.6 million input tokens and between 64,000 and 150,000 output tokens. The engine consumed none. Because one attempt does not predict the next, getting a usable result can take several attempts, each at that cost.

Observation

The agent is not reproducible

Two attempts were run on Caffeine and two on Timefold Solver, from an identical starting point and an identical instruction.

On Caffeine they did not agree. One removed 6 declarations, the other 21. They agree on 5. Both used the same method: an analyser over the compiled class files, built with the ASM library. Both deleted 15 private constructors. The first attempt then put them back, because the project’s build tooling warns about a helper class without a private constructor. The second kept the deletions. That one judgement call accounts for most of the difference between the two answers.

On Timefold Solver they very nearly did agree: 405 and 399 declarations, of which 398 are the same. Both wrote an analysis program that reads compiled class files. Both arrived at much the same answer, including the same mistake on the same two classes, which is the subject of the second example below.

So the agent’s answer depends on judgement calls it makes along the way, even when its method is the same. The engine performs the same computation on the same input and gives the same answer.

Observation

Errors in the agent’s output

Every one of these compiles, so no amount of rebuilding would find them. The harness’s checks found the second and third. QuestDB’s test suite found the first. Timefold Solver’s own build found the second as well.

// io/questdb/cutlass/http/HttpRequestProcessor.java
default short getSupportedRequestTypes() {
    return METHOD_GET;
}

// io/questdb/cutlass/http/HttpPostPutProcessor.java
public interface HttpPostPutProcessor extends HttpRequestProcessor {
    @Override
    default short getSupportedRequestTypes() {
        return METHOD_POST | METHOD_PUT | NON_MULTIPART_REQUEST;
    }
    ...
}

public final class ConstraintCollectors {
    ...
    private ConstraintCollectors() {
    }
}

public final class Joiners {
    ...
    private Joiners() {
    }
}

[ERROR] Failed to execute goal org.revapi:revapi-maven-plugin:0.15.1:check (check)
        on project timefold-solver-core: The following API problems caused the build to fail:
[ERROR] java.method.visibilityIncreased: method void
        ai.timefold.solver.core.api.score.stream.ConstraintCollectors::<init>(): visibility increased
[ERROR] java.method.visibilityIncreased: method void
        ai.timefold.solver.core.api.score.stream.Joiners::<init>(): visibility increased

Conclusion

Codebase-wide changes need an engine behind the agent

An AI coding agent is good at writing new code within a bounded scope: a function, a feature, a test, a fix. These experiments show that too. All six attempts wrote their own analysis programs, unprompted, each of them several hundred lines of working bytecode analysis.

Removing dead code is a different kind of task. It changes the whole codebase at once, every decision in it depends on the rest of the program, and it has to be right in thousands of places. On that task the experiments show three problems:

• It is slow and costly. Claude took 8 to 14 times as long as the engine on all four projects, and consumed 5.2 to 15.6 million input tokens per attempt. The engine consumes none. See The agent costs far more time and tokens.
• It gives a different answer each time. Two attempts on Caffeine, from the same starting point and the same instruction, removed 6 and 21 declarations, agreed on 5, and split on a single judgement call. See The agent is not reproducible.
• Its mistakes pass the build. In five of six attempts it broke a build, a test suite or a public API, and every one of those results compiles. See Errors in the agent’s output.

The agent was not careless. Every attempt ended when it judged itself finished, well inside the two-hour cap. On Timefold Solver the two independent attempts agreed on 398 of about 400 declarations. The problems come from asking it to build, in reasonable time, what a codebase-wide change depends on: every name in the program resolved, and a rule for each way Java code stays alive without being referenced.

In other words, this experiment shows what an agent is not good at, despite being an excellent coder: tasks that require a spotless view of how pieces of code connect and behave, and the ability to make fundamental changes to the code without introducing any errors.

The CodeLaser engine provides that. It gives the same answer every time. Used together, each does the part it is suited to. An agent that calls the engine leaves the code mechanics to it: resolving names, working out what nothing reaches, making the edits. It spends its own time and tokens on deciding what to tackle, judging the findings and explaining the result.

The complete record of every run, from diffs and build logs to session transcripts, is available to anyone evaluating CodeLaser.

Appendix

A. How it was measured

The projects. Four public repositories, each pinned to one commit, each in a sandbox of its own, each reset before every run. The upstream repositories were never written to.

• Caffeine, 07c6e370c
• Timefold Solver, 290c87fc53
• Trino, 27e3b9c8d62
• QuestDB, 714b750076

The file counts are the harness’s own. The line counts are .java lines outside target/ and build/, counted separately, because the harness does not record them.

The agent’s environment. Claude Code ran in a container with network access and a shell, holding a copy of the project and nothing else of the harness. No restriction was placed on what tools it could use: it could write and run any program it liked, in any language on the image.

The harness. For every run, on either side, it:

• resets the project to its pinned commit, and records what it reset to
• builds it with the project’s own command
• runs the engine or the agent
• counts the declarations in the project before and afterwards, with the same parser for both sides, keyed by file, enclosing type and signature
• runs its checks over the result (below)
• only then runs the project’s own test suite, so that each side’s time is its own work
• writes everything it saw to one directory for that run

The checks. Four can fail a run: a surviving class that lost every constructor it declared; an enum that lost a constant while surviving code still reads constant positions; a class emptied of every member that nothing names any more; and an edit to a file the project declares generated-only. Two more are counts recorded beside them, not faults. One of those counts is classes emptied of their members that surviving code still names, as a type in a signature, a supertype or a X.class. Such a class could not have been deleted, so emptying it is the correct outcome. Runs on both sides have them: 18 in the Trino agent run, 2 in the Timefold Solver engine run, 1 in the Trino engine run.

Parameters. The counter records each method parameter as a declaration of its own, so a three-parameter method that goes counts as four. Nearly every removed parameter belongs to a method or constructor that was itself removed. Without parameters, the QuestDB totals are 949 for the engine and 1,322 for the agent.

Files the parser cannot read. The counter parses with JavaParser, which cannot parse 901 of Trino’s files, 146 of QuestDB’s and 13 of Caffeine’s, and none of Timefold Solver’s. No declaration in those files is counted for either side. On Trino, both sides edited three of those files into a form JavaParser can read. That adds declarations to the count after the run and takes none away, so no removal count is affected.

Tokens. Input tokens are the whole context the agent read on each step, nearly all of it re-read from cache. The engine consumes none, because it is a compiled program. It is not free of AI involvement: the script that drives it and the list of roots were produced with Claude Code beforehand. That preparation is paid once, not on every run.

Tests. Every test count in this report comes from one run of the project’s tests on a run’s final result. Tests are counted as JUnit reports them: a parameterised test counts once per parameter set, and tests a project itself marks as skipped are included (in the untouched projects, 7 on Timefold Solver and 470 on QuestDB). On Timefold Solver that run is the whole suite, 6,269 tests in 608 test classes, the same as on the untouched project. On QuestDB it is the whole suite of 42,044 tests in 1,915 classes, except IODispatcherTest and PGJobContextTest, 457 tests between them, which stall under the harness. That leaves 41,587 tests on both sides. On Trino the full suite was not run. The harness runs every test class of each Maven module in which the run changed at least one file. The agent edited 1,402 source files to the engine’s 367, and removed something from 1,211 classes to the engine’s 273. So 2,301 test classes were selected for the agent and 2,026 for the engine. On Caffeine no suite is run, for the reason given with its results.

Appendix

B. Why each root is on the list

60,591 of these are test methods. With every test-related reason included (a class literal in a test, a supertype a runner finds, an inherited test method, an argument provider, a JUnit 3 suite) it is 72,103, or 54 per cent.

Appendix

C. Full figures for every run

Counts are of declarations: types, methods, constructors, fields, enum constants and method parameters, measured by parsing the project before and after each run. A removed method’s parameters are counted with it, so the totals are larger than the number of types, methods and fields removed. Both sides are counted the same way. Appendix A gives the totals without parameters.