Google Summer of Code: Evaluation - Revision history

Zehra: Zehra moved page GSoC: Evaluation to Google Summer of Code: Evaluation: shorthand GSoC doesn't look too good for page titles

2025-12-07T01:06:00Z

Zehra moved page GSoC: Evaluation to Google Summer of Code: Evaluation: shorthand GSoC doesn't look too good for page titles

← Older revision		Revision as of 01:06, 7 December 2025
(No difference)

Zehra: Zehra moved page Google Summer of Code Evaluation to GSoC: Evaluation: let's make Google Summer of Code the main page, with subcategories and pages prefixed with '''GSoC:'''

2025-12-07T01:01:55Z

Zehra moved page Google Summer of Code Evaluation to GSoC: Evaluation: let's make Google Summer of Code the main page, with subcategories and pages prefixed with '''GSoC:'''

← Older revision		Revision as of 01:01, 7 December 2025
(No difference)

Learner: Completely replace the evaluation criteria from 2007. There were too many points of confusion and the criteria were poorly defined. This year their are 6 specific categories.

2008-04-09T05:23:53Z

Completely replace the evaluation criteria from 2007. There were too many points of confusion and the criteria were poorly defined. This year their are 6 specific categories.

← Older revision		Revision as of 05:23, 9 April 2008
Line 1:		Line 1:
	~~The Google mentor evaluation form allows for three positive~~ and ~~two negative~~ evaluation ~~levels (in addition~~ to ~~unevaluated/zero):~~ '''~~+4 +2 +1 -1 -2~~'''		In order to help consistently, effectively, and efficiently evaluate GSoC student applications, specific evaluation criteria should be used by project mentors. GSoC applications are to be scored on a scale of '''0 to 6''' where 0 is a ''poor candidate'' that meets no evaluation criteria and 6 would indicate a ''perfect candidate'' that fulfills all criteria.

	In order to ~~help set an initial sorting of~~ applications, the ~~following guidelines should~~ be ~~used~~ for ranking ~~evaluations~~.		In order to score applications, each mentor needs to indicate the project(s) they would be willing to mentor by selecting the "I am willing to Mentor" button on the mentor dashboard. For scoring an application, apply the below 6 evaluation criteria and evaluate a judgement scoring between 0.0 and 1.0 for each. Round the cumulative score to the closest integer when ranking. '''''Only rank applications you are willing to mentor.'''''

	= ~~Guidelines~~ =		= Evaluation Criteria =
	* Do not evaluate if you are ambivalent or have ''absolutely no intention or interest in interacting with the student on the task'' being proposed. That is, '''if you don't care, then don't evaluate'''.
	* If you do care, especially if you want to be listed as the mentor for a given idea (i.e. you'll willing to help with their final review and evaluation), say so in the comments and '''use the "I'm willing to mentor" button'''.
	* A '''negative evaluation implies that you don't want the proposal to go forward''' completely regardless of any other submissions. Negative means not even minimally desirable or insufficiently proposed to be known.
	* Base rankings on the assertion that there are unlimited slots. That is, '''rank completely independently''' without a consideration that ranking positively might somehow be "stealing" a slot from another proposal.
	* Don't rank negative simply because you don't like an idea as much as others.
	* The numbers will go through a final sort evaluation before being submitted to match up with mentor interests.
	* Evaluations should '''take overall potential impact into (strong) account'''. How many people would the proposal prospectively benefit? Does it have academic/research merit outside of gaming? Academic applicability was one of our primary application acceptance criteria.
	* Proposals that don't actually propose ''any'' work, even general maintenance or work in a vague area, are considered inappropriate and not eligible to be evaluated. If it does propose general maintenance or indicates a vague area of work, then it's merely a negative evaluation.
	* '''Only evaluate once'''. Unless you're changing your evaluation to a different value (like from +2 to +4 or +1 to -1), you shouldn't evaluate multiple times per application. Never evaluate to values other than those predefined (+4, +2, +1, -1, -2).
	* Do not base your evaluation on the evaluations and comments of others. Base the application on its own merit as you understand it and with the responses of the student. Use the criteria contained here.

	~~= Evaluation Levels =~~		* '''Significance:''' Application Impact and Relevance
	* '''~~+4 => EXCEPTIONAL~~'''		** How many people would the proposal prospectively benefit?
	** ~~Exceptionally interesting and/or very appropriate and/or very well thought out~~ proposal ~~with detail.~~		** Does it have benefits outside of our community?
	** ~~Should show an understanding~~ of ~~BZFlag development constraints~~ and~~/or have compelling immediate end-user impact potential.~~		* '''Approach:''' Application Quality and Feasibility
	* '''~~+2 => GOOD~~'''		** How well is their application written?
	** ~~Interesting~~ and~~/or appropriate~~ and~~/or detailed.~~		** Does what they propose seem reasonable?
	** ~~Should show an understanding and be~~ well ~~proposed.~~		* '''Visibility:''' Responsive and Engaging
	* '''~~+1 => MINIMAL~~'''		** Have they been active in the channel?
	** ~~Minimally interesting (i.e. "not undesirable")~~		** Do they respond to questions and engage in discussions?
	* '''~~-1 => POOR~~'''		* '''Charisma:''' Personality and Ease-of-Interaction
	** ~~Mildly inappropriate or~~ is ~~very poorly defined.~~		** Are they easy to talk to?
	** ~~Submission maybe lacks detail on the proposed work.~~		** How well do they interact with others?
	* '''-2 ~~=> UNDESIRABLE~~'''		* '''Potential:''' Long Term Developer Karma
	** ~~The proposal should~~ not even ~~be considered~~.		** Are they excited about working on BZFlag?
	** ~~This should mostly be~~ the ~~case~~ for ~~ideas that~~ are ~~unacceptable to~~ the ~~gameplay even with unlimited developer resources~~.		** Perceived chance that they will stick around and keep coding?
			* '''Ability:''' Technical Coding Aptitude
			** How impressive is their patch?
			** Can they actually write code?

			= Notes =

			The Google mentor evaluation form allows for three positive and two negative evaluation levels (in addition to unevaluated/zero): '''+4 +2 +1 -1 -2'''. The sum of all your rank selections should equal more than 0 and less than 7. Additionally:

			* Do not use negative scores except to correct or adjust your own evaluation.
			* Do not adjust or compensate for other mentor scores (up or down) even to correct an obvious mistake.
			* Do not evaluate them if you don't care for their proposal and/or don't really intend to mentor that student.
			* Do not base your evaluation on the evaluations and comments of others.

			Scores for each application will be '''averaged''' and ''weighted by mentor assignments'' after evaluations are complete and plugged into a spreadsheet for group review before the final ordering.

Learner: don't evaluate based on others

2007-03-27T19:56:37Z

don't evaluate based on others

← Older revision		Revision as of 19:56, 27 March 2007
Line 13:		Line 13:
	* Proposals that don't actually propose ''any'' work, even general maintenance or work in a vague area, are considered inappropriate and not eligible to be evaluated. If it does propose general maintenance or indicates a vague area of work, then it's merely a negative evaluation.		* Proposals that don't actually propose ''any'' work, even general maintenance or work in a vague area, are considered inappropriate and not eligible to be evaluated. If it does propose general maintenance or indicates a vague area of work, then it's merely a negative evaluation.
	* '''Only evaluate once'''. Unless you're changing your evaluation to a different value (like from +2 to +4 or +1 to -1), you shouldn't evaluate multiple times per application. Never evaluate to values other than those predefined (+4, +2, +1, -1, -2).		* '''Only evaluate once'''. Unless you're changing your evaluation to a different value (like from +2 to +4 or +1 to -1), you shouldn't evaluate multiple times per application. Never evaluate to values other than those predefined (+4, +2, +1, -1, -2).
			* Do not base your evaluation on the evaluations and comments of others. Base the application on its own merit as you understand it and with the responses of the student. Use the criteria contained here.

	= Evaluation Levels =		= Evaluation Levels =

DTRemenak: spelling fixes

2007-03-27T17:08:29Z

spelling fixes

← Older revision		Revision as of 17:08, 27 March 2007
Line 4:		Line 4:

	= Guidelines =		= Guidelines =
	* Do not evaluate if you are ambivalent or have ''~~absoluately~~ no intention or interest in interacting with the student on the task'' being proposed. That is, '''if you don't care, then don't evaluate'''.		* Do not evaluate if you are ambivalent or have ''absolutely no intention or interest in interacting with the student on the task'' being proposed. That is, '''if you don't care, then don't evaluate'''.
	* If you do care, especially if you want to be listed as the mentor for a given idea (i.e. you'll willing to help with their final review and evaluation), say so in the comments and '''use the "I'm willing to mentor" button'''.		* If you do care, especially if you want to be listed as the mentor for a given idea (i.e. you'll willing to help with their final review and evaluation), say so in the comments and '''use the "I'm willing to mentor" button'''.
	* A '''negative evaluation implies that you don't want the proposal to go forward''' completely regardless of any other submissions. Negative means not even minimally desirable or insufficiently proposed to be known.		* A '''negative evaluation implies that you don't want the proposal to go forward''' completely regardless of any other submissions. Negative means not even minimally desirable or insufficiently proposed to be known.
Line 22:		Line 22:
	** Should show an understanding and be well proposed.		** Should show an understanding and be well proposed.
	* '''+1 => MINIMAL'''		* '''+1 => MINIMAL'''
	** Minimally interesting (i.e. "not ~~undesireable~~")		** Minimally interesting (i.e. "not undesirable")
	* '''-1 => POOR'''		* '''-1 => POOR'''
	** Mildly inappropriate or is very poorly defined.		** Mildly inappropriate or is very poorly defined.
Line 28:		Line 28:
	* '''-2 => UNDESIRABLE'''		* '''-2 => UNDESIRABLE'''
	** The proposal should not even be considered.		** The proposal should not even be considered.
	** This should mostly be the case for ideas that are ~~unacceptible~~ to the gameplay even with unlimited developer resources.		** This should mostly be the case for ideas that are unacceptable to the gameplay even with unlimited developer resources.

Learner: emphasis

2007-03-27T01:11:30Z

emphasis

← Older revision		Revision as of 01:11, 27 March 2007
Line 4:		Line 4:

	= Guidelines =		= Guidelines =
	* Do not evaluate if you are ambivalent or have absoluately no intention or interest in interacting with the student on the task being proposed. That is, if you don't care, then don't evaluate.		* Do not evaluate if you are ambivalent or have ''absoluately no intention or interest in interacting with the student on the task'' being proposed. That is, '''if you don't care, then don't evaluate'''.
	* If you do care, especially if you want to be listed as the mentor for a given idea (i.e. you'll willing to help with their final review and evaluation), say so in the comments and use the "I'm willing to mentor" button.		* If you do care, especially if you want to be listed as the mentor for a given idea (i.e. you'll willing to help with their final review and evaluation), say so in the comments and '''use the "I'm willing to mentor" button'''.
	* A negative evaluation implies that you don't want the proposal to go forward completely regardless of any other submissions. Negative means not even minimally desirable or insufficiently proposed to be known.		* A '''negative evaluation implies that you don't want the proposal to go forward''' completely regardless of any other submissions. Negative means not even minimally desirable or insufficiently proposed to be known.
	* Base rankings on the assertion that there are unlimited slots. That is, rank completely independently without a consideration that ranking positively might somehow be "stealing" a slot from another proposal. Don't rank negative simply because you don't like an idea as much as others.		* Base rankings on the assertion that there are unlimited slots. That is, '''rank completely independently''' without a consideration that ranking positively might somehow be "stealing" a slot from another proposal.
			* Don't rank negative simply because you don't like an idea as much as others.
	* The numbers will go through a final sort evaluation before being submitted to match up with mentor interests.		* The numbers will go through a final sort evaluation before being submitted to match up with mentor interests.
	* Evaluations should take overall potential ~~'''~~impact~~'''~~ into (strong) account. How many people would the proposal prospectively benefit? Does it have academic/research merit outside of gaming? Academic applicability was one of our primary application acceptance criteria.		* Evaluations should '''take overall potential impact into (strong) account'''. How many people would the proposal prospectively benefit? Does it have academic/research merit outside of gaming? Academic applicability was one of our primary application acceptance criteria.
	* Proposals that don't actually propose any work, even general maintenance or work in a vague area, are considered inappropriate and not eligible to be evaluated. If it does propose general maintenance or indicates a vague area of work, then it's merely a negative evaluation.		* Proposals that don't actually propose ''any'' work, even general maintenance or work in a vague area, are considered inappropriate and not eligible to be evaluated. If it does propose general maintenance or indicates a vague area of work, then it's merely a negative evaluation.
	* Only evaluate once. Unless you're changing your evaluation to a different value (like from +2 to +4 or +1 to -1), you shouldn't evaluate multiple times per application. Never evaluate to values other than those predefined (+4, +2, +1, -1, -2).		* '''Only evaluate once'''. Unless you're changing your evaluation to a different value (like from +2 to +4 or +1 to -1), you shouldn't evaluate multiple times per application. Never evaluate to values other than those predefined (+4, +2, +1, -1, -2).

	= Evaluation Levels =		= Evaluation Levels =

Learner: only evaluate once

2007-03-27T01:06:24Z

only evaluate once

← Older revision		Revision as of 01:06, 27 March 2007
Line 11:		Line 11:
	* Evaluations should take overall potential '''impact''' into (strong) account. How many people would the proposal prospectively benefit? Does it have academic/research merit outside of gaming? Academic applicability was one of our primary application acceptance criteria.		* Evaluations should take overall potential '''impact''' into (strong) account. How many people would the proposal prospectively benefit? Does it have academic/research merit outside of gaming? Academic applicability was one of our primary application acceptance criteria.
	* Proposals that don't actually propose any work, even general maintenance or work in a vague area, are considered inappropriate and not eligible to be evaluated. If it does propose general maintenance or indicates a vague area of work, then it's merely a negative evaluation.		* Proposals that don't actually propose any work, even general maintenance or work in a vague area, are considered inappropriate and not eligible to be evaluated. If it does propose general maintenance or indicates a vague area of work, then it's merely a negative evaluation.
			* Only evaluate once. Unless you're changing your evaluation to a different value (like from +2 to +4 or +1 to -1), you shouldn't evaluate multiple times per application. Never evaluate to values other than those predefined (+4, +2, +1, -1, -2).

	= Evaluation Levels =		= Evaluation Levels =

Learner: /* Guidelines */

2007-03-26T00:48:29Z

Guidelines

← Older revision		Revision as of 00:48, 26 March 2007
Line 10:		Line 10:
	* The numbers will go through a final sort evaluation before being submitted to match up with mentor interests.		* The numbers will go through a final sort evaluation before being submitted to match up with mentor interests.
	* Evaluations should take overall potential '''impact''' into (strong) account. How many people would the proposal prospectively benefit? Does it have academic/research merit outside of gaming? Academic applicability was one of our primary application acceptance criteria.		* Evaluations should take overall potential '''impact''' into (strong) account. How many people would the proposal prospectively benefit? Does it have academic/research merit outside of gaming? Academic applicability was one of our primary application acceptance criteria.
			* Proposals that don't actually propose any work, even general maintenance or work in a vague area, are considered inappropriate and not eligible to be evaluated. If it does propose general maintenance or indicates a vague area of work, then it's merely a negative evaluation.

	= Evaluation Levels =		= Evaluation Levels =

Learner: clarify

2007-03-26T00:45:22Z

clarify

← Older revision		Revision as of 00:45, 26 March 2007
Line 12:		Line 12:

	= Evaluation Levels =		= Evaluation Levels =
	* +4 => EXCEPTIONAL~~: exceptionally~~ interesting and/or very appropriate and/or very well thought out proposal with detail. Should show an understanding of BZFlag development constraints and/or have .		* '''+4 => EXCEPTIONAL'''
	* +2 => GOOD~~: interesting~~ and/or appropriate and/or detailed. Should show an understanding and be well proposed.		** Exceptionally interesting and/or very appropriate and/or very well thought out proposal with detail.
	* +1 => MINIMAL~~: minimally~~ interesting (i.e. "not undesireable")		** Should show an understanding of BZFlag development constraints and/or have compelling immediate end-user impact potential.
	* -1 => POOR~~: mildly~~ inappropriate or is very poorly defined. ~~This~~		* '''+2 => GOOD'''
	* -2 => UNDESIRABLE: should not even be considered. This should mostly be the case for ideas that are unacceptible to the gameplay even with unlimited developer resources.		** Interesting and/or appropriate and/or detailed.
			** Should show an understanding and be well proposed.
			* '''+1 => MINIMAL'''
			** Minimally interesting (i.e. "not undesireable")
			* '''-1 => POOR'''
			** Mildly inappropriate or is very poorly defined.
			** Submission maybe lacks detail on the proposed work.
			* '''-2 => UNDESIRABLE'''
			** The proposal should not even be considered.
			** This should mostly be the case for ideas that are unacceptible to the gameplay even with unlimited developer resources.

Learner: Protected "Google Summer of Code Evaluation": GSoC guidelines are only for mentors [edit=autoconfirmed:move=autoconfirmed]

2007-03-26T00:39:35Z

Protected "Google Summer of Code Evaluation": GSoC guidelines are only for mentors [edit=autoconfirmed:move=autoconfirmed]

← Older revision		Revision as of 00:39, 26 March 2007
(No difference)