referees: (saso 2016)
SASO Referees ([personal profile] referees) wrote in [community profile] sportsanime2016-08-27 04:44 pm
Entry tags:

SASO 2016: Talkback

Talkback


This post is closed; thank you for your feedback. If you would like to provide constructive criticism, please contact the mods directly. All further replies to this entry are screened.

The mods still plan to reply to the topics listed at the bottom of the post.


--

This is not the final wrap-up results/etc. post—that's coming in a couple of days. But while the mods tabulate final scores and create prize graphics, now's a good time to look back and reflect on your SASO 2016 experience.

Please leave comments giving us feedback on the event this year! What did you like? What did you dislike or find frustrating? What can we improve for next year, and how can we improve it? What was your favorite part of the event? Least favorite? Would you participate next year or not? Why? If you could mention your team when making your comment, we'd appreciate it.

If you would prefer to submit your comments in private, you can e-mail us at anime.espn@gmail.com or send us a Dreamwidth PM. We would prefer it if you'd put "Talkback" in the subject line.

Here's what the mod team plans to improve for next year:
  • Voting program reliability. We'd like to apologize again for all of the server dowtnime, and definitely plan to make it much more stable for next year. The voting system in general, though, will remain the same.

  • Bonus round program reliability. We think we got most of the kinks out during this year, but want to keep it stable for next year too.

  • Dreamwidth capability for NSFW cross-posting. We didn't have time to implement it this year, but next year it should be ready to go.

  • Clarifying the rules, especially on what "popular culture media" is acceptable to include in SASO work.
We also wanted to thank you again for being patient with us this year. We had a lot of technical hiccups, and you were all so kind while our heroic codermods worked out the kinks. We really appreciated your support.

--

Mod Replies: (updated 12:55PM EDT 8/28/16)

Please note that inclusion of a topic on this list indicates that we're going to address it, not that we've made any decisions on judging/implementing or not implementing it.

Voting:
The preliminary vote system
Triggers in a participant's assigned vote pool
Juried panel (having the mods decide which entries were the best)
Voting mechanics
Bonus Rounds:
Promptless rounds
One day break between bonus rounds
Bonus round prompt and fill tracking/filtering
Recs round work count
Image size limit
Remix round
Remix/rec round timing
Rec round
Quality control (both prompts and fills)
What is allowed as a fill (tags and ratings)
Unscreening bonus rounds
Fairness to small teams
Comment screening
Main Rounds:
The anonymity policy
Professionalism:
Twitter tone
What information gets posted where
Considering a dreamwidth announcement community
Mod participation in the event
catzuya: (Default)

(frozen comment)

[personal profile] catzuya 2016-08-28 09:06 am (UTC)(link)
More minds brainstorming, more common bugs get seen, that's the beauty of it. If it keeps getting brought up, then obvs there is a problem that needs to be addressed. (If it can't be fixed, explain why, if it can, try.)

Mm, it wasn't too bad honestly? Knowing you had previous experience just tells me you do know why x didn't work and that hs more pull over what I think might work.
Veterans do know what they're talking about.
Elite as it may sound, you guys have experience to back your claims, and that's important in things like this where a mod cannot be around to put out fires. Giving feedback with your name attatched is stressful for some newbies, so knowing a vet can come in and say "oh man one year we did that and it backfired terribly :( it is better though!" reassures me that this has been tried before with bad results.

Maintaining an index Would be good for people trying to find prompts, I saw that was a concern for smaller fandoms trying to fill for something they liked. Accessibility is my main concern for those things, Maintance can be scheduled and divided if you have multiple people working on this, it may not be a current list, but it will be something and that's better than nothing.
catlarks: (SASO: Cards)

(frozen comment)

[personal profile] catlarks 2016-08-28 09:26 am (UTC)(link)
Yesssss, exactly that. It's honestly fascinating to me that like, as a group we get repetitive with certain topics but this just serves to form a consensus? Some of the repeating is riffing on other people's ideas with new ones, some of it is multiple people autonomously suggesting the same thing, and tbh it's really cool just on a "social group" level.

Good to know! I feel like if you do it wrong "pulling rank" about being a veteran and having seen more years of an event can make people feel really bad, but it's also just a useful knowledge base? I've always liked being able to answer friends' little questions about the event so they don't need to bug a mod; I've ingested so many of the events rules to the point of being able to repeat them on command, ahaha.

(I think voting in particular, and MAIN ROUNDS in particular, have undergone the most obvious big changes over the years. Like, in 2013 there were technically four main rounds! It was honestly kind of horrible! Two actually feels so reasonable by comparison.)

I want an index So Bad, it's useful on so many levels. I love the thought of being able to sort for prompts based on several variables, and for fills by that same token. I honestly feel like the biggest drawback of SASO is just being able to FIND all the content for your ship or fandom. (and maintenance does require a large enough crew to split it up into manageable chunks. I sincerely DON'T think it's something mods should focus on, but like. We could do it. We could make a really nice reference thing.)

(in the same vein of old stories: a friend maintained a prompt index in 2014 cross-referenced by character and then ship and it was honestly super thorough and really beautiful, but it took her sooooo much time. Splitting the load is... So helpful...)

I feel like we've halfway wandered off into just chatting and tbh it's sort of nice but here I still am, on the DW post. This is fine.
clefairy: (Default)

(frozen comment)

[personal profile] clefairy 2016-08-28 02:54 pm (UTC)(link)
I didn't see this until after I posted my longass tl;dr but yes, I was of the opinion that a crowdsourced-to-participants index for prompts could probably be pretty easily maintained, even just using googleforms/sheets with options to sort tbh.

also omg I didn't know that MR votes were public last year until I read this feedback thread and saw it brought up multiple times and went back to check voting posts from last year and...that's...hORRIFIC.........
catlarks: (SASO: Cards)

(frozen comment)

[personal profile] catlarks 2016-08-28 05:01 pm (UTC)(link)
Yep, that's how it was last year! I assume a lot of people just don't have context for how voting has evolved, because there isn't really a space in the rules where you can go "and this is what it was like before, and it sucked."

Previously, the mods used DW polls because it was the only option that would verify someone was ACTUALLY signed up for the event; people have suggested other pre-existing options, and that's the problem with them. The voting program used this year was coded specifically for SASO and it just wasn't possible to test it on the scale of the event ahead of time, because where would you even get 300 people to test your program before the event starts? I know I actually think it's REALLY COOL how well the code worked, downtime aside, for this being its ~maiden voyage~

(and I for one very much prefer the opacity of this voting system, versus being able to see the results unfolding live and being able to change your vote in accordance with trends.)
clefairy: (Default)

(frozen comment)

[personal profile] clefairy 2016-08-29 05:30 am (UTC)(link)
Yoo the entire idea that you could change your votes to vote strategically sounds absolutely terrible. If it was going to be a transparent poll then edits shouldn't have been a thing, but that's all a bygone.

The opaqueness is definitely a much better thing right now, though I personally am curious of the final tallies after results are announced just to see how big the gaps were (or how close they were). Though that might just be me and some prefer to live in ignorance.