Google Summer of Code Evaluation

From BZFlagWiki
Revision as of 19:56, 27 March 2007 by Learner (Talk | contribs) (don't evaluate based on others)

Jump to: navigation, search

The Google mentor evaluation form allows for three positive and two negative evaluation levels (in addition to unevaluated/zero): +4 +2 +1 -1 -2

In order to help set an initial sorting of applications, the following guidelines should be used for ranking evaluations.

Guidelines

  • Do not evaluate if you are ambivalent or have absolutely no intention or interest in interacting with the student on the task being proposed. That is, if you don't care, then don't evaluate.
  • If you do care, especially if you want to be listed as the mentor for a given idea (i.e. you'll willing to help with their final review and evaluation), say so in the comments and use the "I'm willing to mentor" button.
  • A negative evaluation implies that you don't want the proposal to go forward completely regardless of any other submissions. Negative means not even minimally desirable or insufficiently proposed to be known.
  • Base rankings on the assertion that there are unlimited slots. That is, rank completely independently without a consideration that ranking positively might somehow be "stealing" a slot from another proposal.
  • Don't rank negative simply because you don't like an idea as much as others.
  • The numbers will go through a final sort evaluation before being submitted to match up with mentor interests.
  • Evaluations should take overall potential impact into (strong) account. How many people would the proposal prospectively benefit? Does it have academic/research merit outside of gaming? Academic applicability was one of our primary application acceptance criteria.
  • Proposals that don't actually propose any work, even general maintenance or work in a vague area, are considered inappropriate and not eligible to be evaluated. If it does propose general maintenance or indicates a vague area of work, then it's merely a negative evaluation.
  • Only evaluate once. Unless you're changing your evaluation to a different value (like from +2 to +4 or +1 to -1), you shouldn't evaluate multiple times per application. Never evaluate to values other than those predefined (+4, +2, +1, -1, -2).
  • Do not base your evaluation on the evaluations and comments of others. Base the application on its own merit as you understand it and with the responses of the student. Use the criteria contained here.

Evaluation Levels

  • +4 => EXCEPTIONAL
    • Exceptionally interesting and/or very appropriate and/or very well thought out proposal with detail.
    • Should show an understanding of BZFlag development constraints and/or have compelling immediate end-user impact potential.
  • +2 => GOOD
    • Interesting and/or appropriate and/or detailed.
    • Should show an understanding and be well proposed.
  • +1 => MINIMAL
    • Minimally interesting (i.e. "not undesirable")
  • -1 => POOR
    • Mildly inappropriate or is very poorly defined.
    • Submission maybe lacks detail on the proposed work.
  • -2 => UNDESIRABLE
    • The proposal should not even be considered.
    • This should mostly be the case for ideas that are unacceptable to the gameplay even with unlimited developer resources.