Package de.cuioss.benchmarking.common.runner

Class AbstractBenchmarkRunner

java.lang.Object
de.cuioss.benchmarking.common.runner.AbstractBenchmarkRunner
public abstract class AbstractBenchmarkRunner extends Object
Abstract base class for JMH benchmark runners that integrates with CUI benchmarking infrastructure.

This runner uses the Template Method pattern to orchestrate benchmark execution while allowing concrete implementations to customize specific steps.

The template method runBenchmark() defines the benchmark execution flow:

  1. Validation of configuration
  2. Preparation phase (prepareBenchmark(BenchmarkConfiguration))
  3. Benchmark execution (executeBenchmark(Options))
  4. Results processing (processResults(Collection, BenchmarkConfiguration))
  5. Cleanup phase (cleanup(BenchmarkConfiguration))

Features:

  • Smart benchmark type detection (micro vs integration)
  • Automatic badge generation with performance scoring
  • Self-contained HTML reports with embedded CSS
  • GitHub Pages ready deployment structure
  • Structured metrics in JSON format

  • Constructor Details

    • AbstractBenchmarkRunner

      protected AbstractBenchmarkRunner()
      Default constructor that initializes the Prometheus metrics manager.

  • Method Details

    • createConfiguration

      protected abstract BenchmarkConfiguration createConfiguration()
      Creates the benchmark configuration for this runner. This method must provide a complete configuration including: - Benchmark type - Include pattern - Result file name and directory - Throughput and latency benchmark names - Any other specific configuration
      Returns:
      the complete benchmark configuration

    • prepareBenchmark

      protected abstract void prepareBenchmark(BenchmarkConfiguration config) throws IOException
      Prepares the benchmark environment. This method is called after configuration validation and before benchmark execution. Use this for initialization tasks like: - Setting up resources - Initializing caches - Configuring logging
      Parameters:
      config - the benchmark configuration
      Throws:
      IOException - if preparation fails

    • executeBenchmark

      protected Collection<org.openjdk.jmh.results.RunResult> executeBenchmark(org.openjdk.jmh.runner.options.Options options) throws org.openjdk.jmh.runner.RunnerException
      Executes the benchmark with the given options. Default implementation uses JMH Runner, but can be overridden for custom execution.
      Parameters:
      options - the JMH options
      Returns:
      collection of benchmark results
      Throws:
      org.openjdk.jmh.runner.RunnerException - if benchmark execution fails

    • processResults

      protected void processResults(Collection<org.openjdk.jmh.results.RunResult> results, BenchmarkConfiguration config) throws IOException
      Processes the benchmark results. Default implementation uses BenchmarkResultProcessor and collects Prometheus metrics.
      Parameters:
      results - the benchmark results
      config - the benchmark configuration
      Throws:
      IOException - if processing fails

    • cleanup

      protected abstract void cleanup(BenchmarkConfiguration config) throws IOException
      Performs cleanup after benchmark execution. This method is called after results processing, regardless of success or failure. Use this for: - Releasing resources - Final metrics collection - Post-processing tasks
      Parameters:
      config - the benchmark configuration
      Throws:
      IOException - if cleanup fails

    • beforeBenchmark

      protected void beforeBenchmark(BenchmarkConfiguration config)
      Hook method called before benchmark execution starts. Override to add custom pre-benchmark logic. Default implementation records benchmark start time for Prometheus metrics.
      Parameters:
      config - the benchmark configuration

    • afterBenchmark

      protected void afterBenchmark(Collection<org.openjdk.jmh.results.RunResult> results, BenchmarkConfiguration config)
      Hook method called after benchmark execution completes. Override to add custom post-benchmark logic. Default implementation logs completion.
      Parameters:
      results - the benchmark results
      config - the benchmark configuration

    • buildCommonOptions

      protected org.openjdk.jmh.runner.options.OptionsBuilder buildCommonOptions(BenchmarkConfiguration config)
      Builds JMH options from the benchmark configuration. This method extracts common option building logic that can be reused or extended.
      Parameters:
      config - the benchmark configuration
      Returns:
      JMH options builder with common settings applied

    • validateConfiguration

      protected void validateConfiguration(BenchmarkConfiguration config)
      Validates the benchmark configuration. Throws IllegalArgumentException if configuration is invalid.
      Parameters:
      config - the configuration to validate
      Throws:
      IllegalArgumentException - if configuration is invalid

    • runBenchmark

      public final void runBenchmark() throws IOException, org.openjdk.jmh.runner.RunnerException
      Template method that defines the benchmark execution flow. This is the main entry point that orchestrates all phases of benchmark execution.
      Throws:
      IOException - if I/O operations fail
      org.openjdk.jmh.runner.RunnerException - if benchmark execution fails