Introduction
The Benchmarking Special Interest Group provides a centralized home within nf-core for performance and accuracy testing across all bioinformatics workflows. While initiated by the development of the nf-core/variantbenchmarking pipeline, this group expands the scope to include any type of method comparison, validation, or resource efficiency study.
Core Objectives
- Software Development: Modularizing benchmarking components in nf-core so other developers can easily plug-and-play validation steps into their pipelines.
- Community Needs Forum: Identifying what researchers need to compare (ranging from metagenomics and spatial transcriptomics to drug response prediction) and prioritizing tool development accordingly.
- Network Syntonization: Keeping nf-core aligned with external benchmarking standards and platforms like NCBench, openEBench, and Open Problems in Single-Cell Analysis.
Activities
Our format revolves around our community blog and a monthly meeting. Each meeting features a 30 minute presentation followed by 30 minutes of round table discussions and feedback on selected topics. Presentations will cover a diverse range of areas, including:
- Benchmarking projects detailing both methods and results
- New software tools in the field
- Infrastructure benchmarking
- Inspiring talks from leading genomics researchers
- Ongoing novel efforts in benchmarking
- Technical sessions providing practical examples of applying nf-core pipelines to benchmarking
Blog Posts
We will use the nf-core blogs to share group updates, benchmarking related projects, summarize technical findings, and highlight community needs.
- Launcing the active group and calling for contributers(TODO)
How to join
Join the nf-core Slack and find us in the #benchmarking channel