I think one of the main issues here is that people are encouraged to review/triage without giving them a good enough introduction on how to do it properly.
My story
I personally wanted to try out the triage feature and it seemed clear enough to me:
- Requires editing: the post needs to be edited for more information (by the author)
- Unsalvageable: the post is spam or not related to the site
Unfortunately, this is not how it is supposed to work at all, and I only found that out when I got review banned.
Additionally, I did not even figure out I was review banned until I manually went to https://stackoverflow.com/review after researching why I have a "bug" that it shows me new reviews are available (the red dot) but then it only shows "There are no review queues available to you". I kept wondering why that bug is happening all the time, and why it keeps showing me the red dot.
I finally went to meta stackoverflow to find out this is a review ban (which most people probably won't even end up researching). And even after I found this out, it took me some more researching to figure out how to find out which review caused the ban (I initially looked into the "Reviews" tab in "All actions" on my profile, there is nothing related to bans/problems with reviews there).
The improvements suggested here should solve the problem of users not knowing when they were banned.
Edit: I also noticed that the red dot keeps re-appearing although there is no way to access the review queues. It would be better if it goes away after you have seen the ban reason.
My suggestion
I totally agree with the suggestions put forward here, and I'd like to add that from a UX perspective it would be better if you:
- Still encourage people to try reviewing out, but give a tutorial first.
- Stage 1: Ideally, use a couple of example posts (where no real reviewing happens!), so that users can practice triaging. If they do it wrongly, tell them what they did wrong and how to improve. This would be an automatic process.
- Stage 2: After users have practiced on enough posts, start showing real posts that others have already reviewed as well. Only let them do a handful of posts here so that moderators can check if they did it correctly, if not, let moderators send them a message on what they did wrongly. This is a manual process, but because each user can only review a handful of posts at first, it will reduce the amount of work.
- Stage 3: Only after users have finished the tutorial (practicing on example posts and real posts), let them do triaging for real. This is where all users start, at the moment. If they still do it wrongly, you can resort to bans at this stage, of course.
I am sorry that I caused more work by wrongly triaging items, but I thought it was clear what to do, but obviously that was my wrong and naive interpretation of the wording.