The BRaliBase dent-a tale of benchmark design and interpretation.
BRaliBase is a widely used benchmark for assessing the accuracy of RNA secondary structure alignment methods. In most case studies based on the BRaliBase benchmark, one can observe a puzzling drop in accuracy in the 40-60% sequence identity range, the so-called 'BRaliBase Dent'. In this article, we show this dent is owing to a bias in the composition of the BRaliBase benchmark, namely the inclusion of a disproportionate number of transfer RNAs, which exhibit a conserved secondary structure. Our analysis, aside of its interest regarding the specific case of the BRaliBase benchmark, also raises important questions regarding the design and use of benchmarks in computational biology.
21 references, click to expand
-
Accurate multiple sequence-structure alignment of RNA sequences using combinatorial optimization.BMC Bioinformatics. 2007 Jul 27;8:271 PMID: 17662141
-
Inferring noncoding RNA families and classes by means of genome-scale structure-based clustering.PLoS Comput Biol. 2007 Apr 13;3(4):e65 PMID: 17432929
-
Specific alignment of structured RNA: stochastic grammars and sequence annealing.Bioinformatics. 2008 Dec 1;24(23):2677-83 PMID: 18796475
-
A comprehensive comparison of comparative RNA structure prediction approaches.BMC Bioinformatics. 2004 Sep 30;5:140 PMID: 15458580
-
Multiple sequence alignment modeling: methods and applications.Brief Bioinform. 2016 Nov;17 (6):1009-1023 PMID: 26615024
-
Fine-tuning structural RNA alignments in the twilight zone.BMC Bioinformatics. 2010 Apr 30;11:222 PMID: 20433706
-
DAFS: simultaneous aligning and folding of RNA sequences via dual decomposition.Bioinformatics. 2012 Dec 15;28(24):3218-24 PMID: 23060618
-
Rfam: an RNA family database.Nucleic Acids Res. 2003 Jan 1;31(1):439-41 PMID: 12520045
-
A benchmark of multiple sequence alignment programs upon structural RNAs.Nucleic Acids Res. 2005 Apr 28;33(8):2433-9 PMID: 15860779
-
Alignment of RNA base pairing probability matrices.Bioinformatics. 2004 Sep 22;20(14):2222-7 PMID: 15073017
-
BAliBASE: a benchmark alignment database for the evaluation of multiple alignment programs.Bioinformatics. 1999 Jan;15(1):87-8 PMID: 10068696
-
An enhanced RNA alignment benchmark for sequence alignment programs.Algorithms Mol Biol. 2006 Oct 24;1:19 PMID: 17062125
-
BAliBASE 3.0: latest developments of the multiple sequence alignment benchmark.Proteins. 2005 Oct 1;61(1):127-36 PMID: 16044462
-
Chaining sequence/structure seeds for computing RNA similarity.J Comput Biol. 2015 Mar;22(3):205-17 PMID: 25768236
-
ExpaRNA-P: simultaneous exact pattern matching and folding of RNAs.BMC Bioinformatics. 2014 Dec 31;15:404 PMID: 25551362
-
A max-margin model for efficient simultaneous alignment and folding of RNA sequences.Bioinformatics. 2008 Jul 1;24(13):i68-76 PMID: 18586747
-
SPARSE: quadratic time simultaneous alignment and folding of RNAs without sequence-based heuristics.Bioinformatics. 2015 Aug 1;31(15):2489-96 PMID: 25838465
-
Fast pairwise structural RNA alignments by pruning of the dynamical programming matrix.PLoS Comput Biol. 2007 Oct;3(10):1896-908 PMID: 17937495
-
Complete probabilistic analysis of RNA shapes.BMC Biol. 2006 Feb 15;4:5 PMID: 16480488
-
Consensus shapes: an alternative to the Sankoff algorithm for RNA consensus structure prediction.Bioinformatics. 2005 Sep 1;21(17):3516-23 PMID: 16020472
-
Pure multiple RNA secondary structure alignments: a progressive profile approach.IEEE/ACM Trans Comput Biol Bioinform. 2004 Jan-Mar;1(1):53-62 PMID: 17048408
Contact
No. 2 Wenbo Road, Zhangqiu District, Jinan, Shandong
Qilu Normal University · Genelibs Bioinformatics Lab
750 Shunhua Rd, Jinan
2F, Bldg F, University Science Park
Tel: 0531-88819269
WeChat Official Account
Follow our WeChat subscription account for real-time updates and the latest in medical and biological research.
Business Email
E-mail: [email protected]
Copyright © 2026.Genelibs Jinan Honghui Biotechnology Co., Ltd. 鲁ICP备20007159号-2
