Comment by tsumnia

Speaking as a CS Education reviewer, some of the criteria can be "signing up to review", though solicitation is often sent to professionals in the domain (through personal requests or blanket email campaigns), as well as through respective mailing lists. I review papers for I think 4-5 conferences, mostly because I have colleagues that serve/publish in those spaces (you declare conflicts of interest to avoid bias).

Each publisher/conference have their own reviewing guidelines to follow, but at least for the conferences I've reviewed for they include: a summary (2-5 sentences tops), the strengths of, the weaknesses of the research, and potentially your opinion on the piece. You are typically asked to include your familiarity with the research space since you may be reviewing methodologies that you were not explicitly trained in. This all distills into a metric that effectively reflects "this paper should be accepted/not accepted" which is then handed to a 'senior' reviewer to summarize for the conference to decide. All of my conferences are double-blind single submission, but I have colleagues that are able to respond to reviewer critiques.

Most conferences recognize things like grammatical issues can happen, so reviewers are asked to only point them out rather than use them as a basis for rejection; however if the paper is riddled with mistakes, then it can be grounds for rejection. Likewise, since CS Education is a combination of CS and cognitive psychology, some of the discussion can be attributed toward "appropriateness for CS education research". For example, I once reviewed a paper that clearly was including theater-based education techniques but had CS shoehorned in one paragraph (that was it). Alternatively, measuring time delays in student responses to a tutoring system can help distinguish when students become distracted or take a break.