Sun Dec 12 10:20:20 PST 2004 Kevin Karplus This file contains comparison with other predictors, based on the information from http://predictioncenter2.llnl.gov/casp/casp6 Note: tables are recomputed on the fly, and take forever to reload with the browser. Levels: top (the best prediction, as far as I can tell) very good (one of the top few) good (near the top, but predictions so similar that they are hard to distinguish) respectable (in the top group when there are many similar top ones) average (near the middle) poor (in the lower group, when some predictors did well) T0196 CM respectable Model 3 is listed as 44th in GDT, but differences among top groups are small. GDT plot looks like we should be higher than 44, but it may be that the curves all pile up. There is something wrong with the tables on the CASP web site, since T0196TS331_1 (hmspectr3) has a GDT of 80.62% but only has 71.9% of the evaluable residues, and gets an RMSD of 5.51 for them. T0197 FR very good model 1 ninth in GDT score, other 8 clearly better in GDT curves. (4 from Baker's group, plus first models from 223, 213, 210, 035). T0198 FR average model 5 (automatic prediction) is 139 out of 454 models. Baker's group nailed this one, but we failed to form the 6-helix bundle, though we knew that was what was needed. T0199 multi-domain top Our model 1 had the best GDT for the whole protein, but that may not be very meaningful, since it was still only 22.21% T0199_1 CM very good Our model 1 had second highest GDT score, but it looks to me like there are 4 or 5 GDT curves that are better than ours. Small differences in GDT are not very meaningful around 81%. Our CA RMSD for full-length domain is about 10th. T0199_2 FR very good Our models 3,2,1 are 3rd through 5th in GDT score. Ginalski (450) clearly beat us, and some others are comparable to us. T0199_3 FR very good Our model 4 is 7th in GDT score, and our model 1 is 18th. T0200 CM respectable Our model 1 is 61st out of 448. T0201 NF very good Our model 1 is 10th in GDT score. We were clearly beaten by Kolinksy and Bujnicki (21) and somewhat beaten by a few others, particularly at the 2-4 Angstrom range. T0202 multidomain poor Our model 1 was 260 out of 444. Most models were poor, but groups Kolinksy and Bujnicki (21), 35, and Ginalski (450) did well. T0202_1 FR poor Our model 1 was 257 out of 442. Many groups did reasonbly, and Ginalski (450) did particularly well. T0202_2 NF poor Our model 3 was 80 out of 393, which sounds ok, but the top 16-20 models were good and the rest were poor. The top groups were Kolinksy and Bujnicki (21), Genesilico (35), Ginalski (450), Baker (100), and 51. T0203 FR respectable Our model 4 (an alignment) was 38 out of 365. We did not have a better model to submit. T0204 CM good Our model 5 was 59/368, our top alignment. If we had been able to complete this alignment, we would have done better---our try4-opt1 (not submitted) had RMSD_CA of only 3.9467, comparable to the top 4 or 5 models. Why were our other models further from the correct one? T0205 CM respectable Our model 4 was 180/387, but model 1 as down around 230. Many groups did better, and Ginalski did great. T0206 FR average Our model 5 was 76/445 and many groups did substantially better. The collagen-like portion was not evaluated. We probably spent too much time trying to trimerize, and did not choose the best templates to work from. T0208 CM good Our model 4 (full auto) is 27/414 but our model 1 is only respectable at 94/414. How did we mess up the full auto? (alignments were not as good) T0209 multidomain good model 3 was 12th, but no one got much on whole chain. T0209_1 FR average model 3 was 91/436 T0209_2 NF very good model 3 was 8/384. model 1 was slightly worse at 16th. T0211 CM respectable model 5 (second alignment) was 60/447 but model 1 was 166/447. Why did we mess up the alignment? T0212 FR good Model 1 was ranked 20, but models from Baker (100), 272, 003, and 176 were substantially better. T0213 FR respectable Model 5 was 54/498, alignment to 1t62A, as suggested by Murzin. Model 1 was 95/498. It isn't clear why we messed up the good alignment. T0214 FR good Models 2 and 1 are 21 and 22/479. PsWatch (685) really nailed this one, getting a 2.11 RMSD model. How? T0215 FR good Model 1 is 33/479 Scheraga's group (64) did the best on this small all-helical protein. T0216 multi-domain good Model 4 is 8th, but all models are so poor this might be meaningless. T0216_1 NF respectable. Model 2 is 36th. No models are good. T0216_2 NF respectable. Models 2 and 1 are 51st and 52nd. No models are good. T0222 multi-domain good Models 3 and 1 are 41 and 42. T0222_1 CM good Models 3 and 1 are 43 and 44. T0222_2 FR good Models 3,2,1 are 16,17,18. T0223 multidomain respectable Model 5 is 126th, but top 10 or so groups much better. T0223_1 CM average. Model 5 is 141/415, model 1 is 228/415. Why did we botch it? T0223_2 FR average. Model 3 is 71, model 1 is 137. Ginalski nailed this one. T0224 FR good Models 3,2,1 are 43,44,45. T0226 multidomain good Model 1 is 63/429. T0226_1 CM good Model 5 is 46/416 Model 1 is 54th, but gets a few more residues at the cost of a little quality. T0226_2 CM respectable Models 4,3,1 are 111th,112th,113th. T0227 FR respectable Model 1 is 82nd, but Genesilico (35) nailed it. T0228 multidomain respectable Model 3 is 109/448 model 5 is 117th, model 1 is 156th. T0228_1 FR respectable Model 3 is 122nd, model 1 is 154th. The full-auto prediction (model 4) is poor. T0228_2 FR respectable? average? GDT plot is missing. Model 3 is 104/426, model 1 is 151st. T0229 multi-domain good Models 3,2,1 are 82,83,84. Lots of groups did well on this one. T0229_1 CM very good (top?) (only 24 residues!) Models 4,3,2,1 are 6,7,8,9. We had 0.86 Ang RMSD-CA, and the top group had 0.75. GDT is not a good measure for models that are this accurate. Really want all-atom RMSD, which casp did not compute. T0229_2 CM very good (top?) Model 4 (full auto) is 12, models 3,2,1 are 71,72,73. Top few models are nearly indistinguishable by GDT. Our model 4 has lowest RMSD. Why did we do (slightly) worse than our full auto? T0230 FR very good Model 1 is 14th. Top models are Skolnick's (are they getting a bonus from compression?) T0231 CM good Model 5 is 50th. Model 1 is 127th. GDT is not a good measure for models this good. Many did well on this target. Our model 1 is better on rmsd_ca than model 5. On RMSD for full-length models, we were 12th. T0232 multidomain very good top? models 3,2,1 are ranked 3,4,5, but I think model 1 is the best of these. T0232_1 CM respectable model 3 is 93/456. T0232_2 CM very good models 2,1 are 4th and 5th. Ginalski (450) did slightly better with model 1, and a couple of groups had a better models than us (model 2 for 579 and model 3 for 604). T0233 multi-domain good Model 4 is 56, model 1 is 66. Several groups had good models, top groups almost indistinguishable by CASP6 measures. T0233_1 CM good Model 3 is 69th, Models 2 and 1 are 116th and 117th. Top groups almost indistinguishable by CASP6 measures. T0233_2 CM good Model 4 is 50th, models 2 and 1 are 102nd and 103rd. Model 4 is very good out to 84% of residues, then rapidly goes bad. Models 1 and 2 are moderately good out to 95% of residues. GDT score does not capture this behavior well. model 4 has 1.81 Ang RMSD-CA, but only over 95% of residues. Models 1 and 2 have 1.97 Ang RMSD-CA over all residues. Top models get down to 1.67 Ang RMSD-CA. T0234 CM very good Models 2,3,1 in ranks 4,5,6. GeneSilico's model 1 noticeably better. T0235 multidomain good Model 3 is 43, model 1 is 52. TO235_1 CM good Model 3 is 50th, model 1 is 60th. TO235_2 FR respectable Model 3,2,1 are 31st, 32nd, 33rd. T0237 multi-domain good Model 1 is 6th, but no one did well on whole chain. T0237_1 FR good Model 1 is 21st. T0237_2 FR good Model 3 is 11th, model 1 is 30th. T0237_3 FR very good top? Model 1 had best GDT score, but CLB3group (94) had better RMSD-CA, and did not deviate as much after 80% of chain matched. T0238 NF good Model 1 is 14th. Proteinshop (174) did substantially better. T0239 FR average Model 1 is 131st. T0240 CM very good Model 2 is 4th, model 1 is 123rd. (Should have guessed monomer first) Ginalski (450) had better core, and group 21 had good core without deviating too far anywhere (good RMSD). T0241 multidomain respectable model1 is 21st. No one has good whole-chain model. T0241_1 NF very good model 4 is 12th, model 1 is 20th. T0241_2 NF good model 1 is 21st. T0242 NF respectable model 5 is 112th, model 1 is 127/475. T0243 FR average models 4,1 are 110th and 111th. T0244 CM good? model 4 is 21st, model 1 is 229th. The alignments get good results for 70% of residues, the final model gets good results for 90% of residues, but not as good for the best 70%. What did we throw away in quality to pick up the extra 20% at larger distance? Skolnick's group seems to have gotten good results all the way out to 90%. Did we just end up with a poor choice of template? T0246 CM respectable. model 1 is 188th. Did we choose the wrong template? T0247 multidomain respectable. model 5 is 72nd, model 1 is 180th T0247_1 CM respectable/average model5 is 64th, model 1 is 196th. The alignment in model5 is respectable, but we messed it up a lot in getting model 1. T0247_2 CM good. model 3 is 36th, model 1 is 78th. T0247_3 CM very good model 4 is 7th, model 1 is 40th. T0248 multidomain very good top? model 1 is 3rd gdt, best rmsd. T0248_1 FR good model 1 is 14th. T0248_2 NF good model 1 is 21st. T0248_3 FR respectrable. model 1 is 51st T0249 multidomain, very good model 4 is 5th. model 1 is 10th. T0249_1 FR very good model 2 is 5th, model 1 is 24th. T0249_2 FR respectable model 5 is 31st, model 1 is 92nd. T0251 FR respectable. model 1 is 84th. T0262 multidomain, average. model 3 is 92nd, model 1 is 226th. T0262_1 FR good/respectable model 3 is 32nd, model 1 is 165th. Why is model 3 so much better than model 1? T0262_2 FR average. model 4 is 98th, model 1 is 158th. Sun Dec 19 08:55:39 PST 2004 Kevin Karplus T0263 FR respectable model 4 is 41, model 1 is 77 out of 465 T0264 multidomain average model 3 is 158, model 1 is 165 T0264_1 CM respectable model 3 is 90, model 1 is 130 Model 1 traded off some very good residues to get a more complete adequate model. T0264_2 CM very good model 1 is 10th. Kolinski & Bujnicki's model 2 is noticeably better (other top models very similar to our GDT curve). T0265 CM respectable model 3 is 89th, model 1 is 106th. T0266 CM good model 2 is 18th, model 1 is 60th. Note: smooth GDT scores prefer our model1 to model2, so the differences here are in the measurement noise of the assessment. T0267 CM good model 5 is 22nd, model 1 is 44th. Skolnick-Zhang's model 5 has noticeably better GDT curve (how much is that better model and how much is artifact from compression?) T0268 multidomain good model 5 is 68th, model 1 is 187th. Our first alignment is pretty good, but we messed it up---why? T0268_1 CM very good model 4 is 42nd, model 1 is 98th. Top groups almost indistinguishable by GDT. Model 4 is full-auto backbone (repacked by rosetta). T0268_2 CM very good/average model 4 is 7th, model 1 is 215th. We had some very good models of this domain (including the full-auto backbone for models 3 and 4), but messed them up in model 1---why? We did almost no work on this one, but I incorrectly chose try3 over try1. T0269 multidomain respectable model 3,1,2 are 144th, 145th, 146th T0269_1 CM very good models 3,2,1 are 52,53,54 top models almost indistinguishable by GDT T0269_2 CM very good models 3,2,1 are 20,21,22 These models are the most complete ones (lowest distance cutoff for 65-95% of residues), but Ginalski has much better fit for up to 60% of residues. T0271 CM good model 3 is 28th, model 1 is 96th. GeneSilico group has noticeably better models. T0272 multidomain respectable model 3 is 48th, model 1 is 169th. No one did well on whole chain, but Baker's group did noticeably better than everyone else. T0272_1 FR average model 3 is 86th, model 1 is 169th. Baker's group did well on this domain. T0272_2 FR respectable/average model 2 is 44th, model 1 is 223rd. (Again the top 2 models were from Baker's group.) T0237 NF good model 1 is 28th. Baker's group did much better. T0274 CM average. model 5 is 129th, model 1 is 233rd. T0275 CM very good models 3,2,1 are 4th, 5th, 6th. T0276 CM good models 2,1 are 25th, 26th Top models almost indistinguishable by GDT. T0277 CM very good (top?) model 3 is 2nd. models 2,1 are 52nd and 53rd. Model 3 has lowest CA RMSD (1.459 Ang). Actually, try1-opt1-scwrl (not submitted) was better model, with 2.419 Ang all-atom rmsd (vs. 2.4813 for model 3). T0279 multidomain respectable model 4 is 120th, model 1 is 146th. T0279_1 CM respectable Models 2,1 are 133rd, 134th. T0279_2 CM respectable models 3,2,1 are 131,132,133. T0280 multidomain respectable. model 1 is 113th. T0280_1 CM respectable. model 4 is 99th, models 2,1 are 119th, 120th. Model 4 is an alignment and has better fit but for less of model (crossover around 75% of residues). T0280_2 FR average model 3 is 198th, model 1 is 217th. Poor performance on this inserted domain probably damaged the surrounding domain. T0281 FR good model 1 is 19th. Baker and Jones had much better models. T0282 CM average model 5 is 115th, model 1 is 245th Skolnick-Zhang had top model, based on template 1cevA, 1gq6A: our top two templates were 1gq6A and 2cevA. Did we include too many templates, diluting the signal? Or did we have bad alignments? our best model was from alignment to 2cevA, so perhaps our alignments weren't good enough.