Engine Performance Metrics

Core performance metrics of a Xiangqi engine:

1. NPS (Nodes Per Second): Indicates the engine’s search speed
2. Average search depth: Indicates the engine’s search depth per unit time
3. Cache hit rate: Indicates transposition table utilization efficiency
4. Pruning rate: Indicates the effectiveness of various pruning techniques
5. ELO rating: Indicates the engine’s strength level

Volume XXXII: Best Practices for Xiangqi Engine Testing

Test Environment Setup

Environmental setup specifications for Xiangqi engine testing:

1. Use the same hardware and operating system
2. Use the same compiler and compilation options
3. Use the same opening book and time control
4. Control processor core isolation (avoid interference from other processes)
5. Record environmental information for result reproduction

Test Game Arrangement

Specifications for arranging Xiangqi engine test games:

1. Opening selection: Use diverse openings (or standard opening test suites)
2. Color rotation: Alternate Red and Black between the test engine and reference engine
3. Time control: Use standard time control (e.g., 15s + 0.5s)
4. Number of games: At least 1000 games to obtain statistically significant results
5. Result recording: Record the result of each game (win/loss/draw)

Result Analysis Methods

Methods for analyzing Xiangqi engine test results:

1. Win rate calculation: Win rate = Number of wins / Total number of games
2. ELO calculation: ELOd = -400 * log10(1/win_rate - 1)
3. Confidence interval calculation: Using normal distribution or Bootstrap methods
4. SPRT decision: Using SPRT to determine whether a modification is effective

Visualization of Test Results

Methods for visualizing Xiangqi engine test results:

1. ELO trend chart: Display ELO changes of the engine over time
2. Win rate comparison chart: Bar chart showing win rate comparisons under different settings
3. Search depth distribution chart: Display the engine’s search depth distribution
4. Time usage distribution chart: Display the engine’s thinking time distribution

Volume XXXIII: Detailed Supplement to the Xiangqi Engine Historical Timeline