Evaluating TeamCity performance between different versions

Question

Is there a recommended way to evaluate the performance impact of a TeamCity upgrade? Is there any way to compare the performance of a new TeamCity version installed on a separate server with the current production environment? In particular:

1. Are there any official performance benchmarks or comparison metrics between the TeamCity versions?
2. Are there any recommended methods or tools to measure TeamCity server performance before and after an upgrade?
3. Are there any known changes in performance-related features or architecture between these versions that may impact build times, agent utilization, or UI responsiveness?

Answer

1. There are no official benchmarks or comparisons between versions. Performance is heavily impacted by a large number of variables that are outside TeamCity's control, so providing some generic numbers could be misleading and contradict observations. For example, plugins, subsystems, and services used, all have an impact on performance
2. There are some recommended approaches to measure performance:
   • The most basic one is to monitor the Administration > Diagnostics page under some known level of the workload. It shows CPU and memory usage, and it can be checked for many other stats, like VCS status, metrics, etc.
   • There is a non-bundled Benchmark plugin. The following blog post provides details on it: Benchmarking TeamCity. Since it runs in the actual environment, it will provide the most accurate information. However, it will test throughput, meaning that it's not recommended to use it on the production server during the actual workload, as it may affect performance. It is recommended to use it on an identical test server.
3. Almost every new release includes performance improvements, so it is recommended that you stay up to date with the release and upgrade notes for each new TeamCity version.