CAMEO model quality estimation (CAMEO QE) is intended to continuously evaluate the performance of local model quality assessment programs (MQAPs). The participating groups are, then, allowed to register their method either as Server or as Standalone.

The CAMEO QE targets consist of all the models returned by the public servers participating to the CAMEO 3D category.

Residues in QE targets are classified as "correct" and "incorrect" applying a lDDT-local score threshold of 60 [ref.: PMID: 23986568]. Based on this assignment, the performance of the various local MQAPs are evaluated through ROC and Precision vs Recall analysis being those metrics independent on the exact nature of the different MQAP functional forms. (more)

Dataset

The assessment is independently made on 4 different datasets of CAMEO QE targets created according to the 'global quality' of the structures expressed as lDDT global score. Namely:

• All: the overall CAMEO QE target dataset
• Low quality: CAMEO QE targets having a lDDT global score between 0 and 40
• Medium quality: CAMEO QE targets having a lDDT global score between 40 and 80
• High quality: CAMEO QE targets having a lDDT global score between 80 and 100

Prediction/Submission format

The coordinates of the CAMEO QE target structures are provided in PDB format. Typically in experimental structural biology, the B-Factor field is used to express the uncertainty of the coordinates. Thus, for CAMEO-QE we propose to use the B-Factor field to specify the expected local atomic coordinate of a model. Specifically:

• Local error estimates should be assigned to all atom coordinate records present in the model
• Errors should be expressed as positive distance in Å between the model atom and the corresponding atom in the reference structure. Negative values are considered as “non-predictions”.
• Error estimates should be inserted into the B-factor field (columns 61-66 in PDB format) without changing any other field in the provided model coordinates. (Remarks and other information can be added using the REMARK fields)
• Please do not add or remove parts of the target coordinates

If you are interested in registering your MQAP to CAMEO QE, feel free to validate the submission mechanisms and prediction format using this tool.

The Baseline predictor

Our baseline predictor is based on sequence conservation. This information is extracted from the PSSM (Position Specific Scoring Matrix) generated by 3 iterations of PSI-BLAST [ref.:PMID: 9254694]. PSI-BLAST is launched using the target sequence as query against and the most recent version of NCBI NR as database.