What is Mutahunter?
Mutahunter is an open-source, language-agnostic tool for LLM-based mutation testing, a way to check whether a test suite detects changes to source code. Developers provide a source file, its test file, and a test command, then run the tool from the command line. It uses a selected language model to create mutants and reports how the tests respond; Python 3.11 or later is required, and the project is licensed under AGPL-3.0.
Mutahunter Features:
Mutant Generation: Mutahunter uses a language model to create altered versions of a selected source file, called mutants, for testing.
Test Command: A command-line option lets developers specify the command that runs their project's tests, such as mvn clean test.
File Selection: Separate source-path and test-path options identify the code to mutate and the test file to run against it.
Model Choice: The run command accepts a model selection, with GPT-4o-mini shown in the setup example and an OpenAI API key used for access.
Mutant Results: The run output classifies mutants as killed or surviving and also reports mutants that time out or encounter compile errors.
Coverage Summary: A final report gives mutation coverage and totals for mutants, survivors, killed mutants, timeouts, compile errors, and run cost.
Mutahunter Benefits:
Test Gaps: Surviving mutants help developers spot code changes their current tests did not detect, giving them concrete areas to investigate.
Focused Test Work: Results tied to specific mutants can help teams decide where additional or stronger assertions may be useful.
Existing Workflow: Using a project's own test command lets developers evaluate test behavior without replacing their established test runner.
Clear Run Review: Separate outcome counts and a coverage figure make it easier to review what happened in a run than inspecting every mutant manually.
Use Cases:
Java Test Review: Run mutation testing on a Java source file and its test file using a Maven command such as mvn clean test.
Weak Assertion Checks: Use surviving mutants while reviewing a test suite to identify places where assertions may not catch changed behavior.
Test Suite Evaluation: Check how effectively an existing suite detects code changes before deciding whether its tests need further attention.
Mutation Run Reporting: Review killed, surviving, timed-out, and compile-error mutants when assessing the outcome of a mutation-testing run.