To make sure that the results in Section 5 were not just training on noise in the data, but were picking up phenomena that should generalize, regularizers were created using the same methods on a subset of the data, and tested on a disjoint subset.
The BLOCKS database was divided into three disjoint sets, with about 10% of the blocks in set 10a, 10% in 10b, and the remaining 80% in 80c. Regularizers were created separately for each of the three sets, and tested on the other two. The ordering of the methods produced by these tests was almost identical to the ordering produced by the self-test presented in Section 5. This separate train-test evaluation lends some extra confidence to the comparative evaluation of the regularizers, and some assurance that the good regularizers will generalize to similar multiple alignments, but little new information, and so the results will not be presented in detail here.