referees: (saso 2016)
SASO Referees ([personal profile] referees) wrote in [community profile] sportsanime2016-08-27 04:44 pm
Entry tags:

SASO 2016: Talkback


This post is closed; thank you for your feedback. If you would like to provide constructive criticism, please contact the mods directly. All further replies to this entry are screened.

The mods still plan to reply to the topics listed at the bottom of the post.


This is not the final wrap-up results/etc. post—that's coming in a couple of days. But while the mods tabulate final scores and create prize graphics, now's a good time to look back and reflect on your SASO 2016 experience.

Please leave comments giving us feedback on the event this year! What did you like? What did you dislike or find frustrating? What can we improve for next year, and how can we improve it? What was your favorite part of the event? Least favorite? Would you participate next year or not? Why? If you could mention your team when making your comment, we'd appreciate it.

If you would prefer to submit your comments in private, you can e-mail us at or send us a Dreamwidth PM. We would prefer it if you'd put "Talkback" in the subject line.

Here's what the mod team plans to improve for next year:
  • Voting program reliability. We'd like to apologize again for all of the server dowtnime, and definitely plan to make it much more stable for next year. The voting system in general, though, will remain the same.

  • Bonus round program reliability. We think we got most of the kinks out during this year, but want to keep it stable for next year too.

  • Dreamwidth capability for NSFW cross-posting. We didn't have time to implement it this year, but next year it should be ready to go.

  • Clarifying the rules, especially on what "popular culture media" is acceptable to include in SASO work.
We also wanted to thank you again for being patient with us this year. We had a lot of technical hiccups, and you were all so kind while our heroic codermods worked out the kinks. We really appreciated your support.


Mod Replies: (updated 12:55PM EDT 8/28/16)

Please note that inclusion of a topic on this list indicates that we're going to address it, not that we've made any decisions on judging/implementing or not implementing it.

The preliminary vote system
Triggers in a participant's assigned vote pool
Juried panel (having the mods decide which entries were the best)
Voting mechanics
Bonus Rounds:
Promptless rounds
One day break between bonus rounds
Bonus round prompt and fill tracking/filtering
Recs round work count
Image size limit
Remix round
Remix/rec round timing
Rec round
Quality control (both prompts and fills)
What is allowed as a fill (tags and ratings)
Unscreening bonus rounds
Fairness to small teams
Comment screening
Main Rounds:
The anonymity policy
Twitter tone
What information gets posted where
Considering a dreamwidth announcement community
Mod participation in the event
catzuya: (Default)

(frozen comment)

[personal profile] catzuya 2016-08-28 01:16 am (UTC)(link)
First, I'd like to say that even with all the hiccups that happened, I did have fun and I was able to collaborate with a group of amazing people. I wouldn't change that for the world.

Now for feedback. I was more than disappointed honestly. While it was a fast paced competitive event that created a fun environment full of memes and put out top notch entries that were amazing to look at, the lack of organization overshadowed that for me.

SASO, "sports anime summer olympics" is a competition based fandom event. And with that, I expected a higher standard of control for this type of event. I want to make it clear that I am in no way blaming the code mod for this, but there should have been more communication from the main moderators to the public. If mods knew that problems were happening and kept happening, please make a universal announcement for all participants to know about it.

Major kudos to code mod who had to work through all of this, i know it must of been tough for you.

However, in the spirit of making things nonbiased, the voting turned out to be the major downfall. With a system meant to be random, people who did go through all the entries prepared to vote for their choices, couldn't, if they had a bad pull. It became unfair in all its fairness. I'm more than positive that Teams most certainly lost votes because of this.

BR commenting.
Im guilty of not really participating in the comment rounds because (this is personal opinion) but i am on my phone 90 percent of the time and it is not very mobile friendly. Not only that, but there is no way to filter through all the entries. A "top-level comment' can only do so much, but when you're shifting through 23 pages and over 1000 comments? I'm sorry.

With all of that, I have a few suggestions.
-Change the voting system to a Juried Panel.
There seems to be enough mods to do this.
Final rounds can be a combination of Mod Votes and Fan Votes. (i have a handy dandy google doc with further information bc I'm Just Like That if anyone is interested in looking) But honestly, the voting system needs to be changed and I hope that the mods consider this. It wasn't downtime that turned people off, it was the system.

-Cap the Bonus Rounds
others have commented on how to improve but, for me personally, quantity does not not mean quality.

-Make DW comms for each bonus round for easier tagging.
(individual posts means people can use tag and fill easier)
Creating a community will allow participants to add their own tags based on ship, bait, tw/cw , filled/unfilled and anything else. Fills can be commented directly into the post and it wont clutter up pages and pages of comments.

Personally for me, things need a major overhaul. But as it stands, I probably won't participate unless its grandstand to give prompts for my friends to fill.

catlarks: (SASO: Cards)

(frozen comment)

[personal profile] catlarks 2016-08-28 01:41 am (UTC)(link)
Mostly these are suggestions for the mods to evaluate so it doesn't matter if I disagree with them, but FWIW bonus rounds HAVE been capped in the past, and I vehemently disagree with the thought that the most creative and productive part of SASO should be limited.

IMO the event is a two-sided coin, geared towards getting its members to improve as creators. Main rounds involve specific limits and challenge you to work within those restrictions to create something as polished (and "quality") as possible. Whereas bonus rounds subscribe to the belief of turning off your inner editor and just CREATING, as much as possible, to get better. Quantity is literally the goal. Capping the bonus rounds does members a disservice.
catzuya: (Default)

(frozen comment)

[personal profile] catzuya 2016-08-28 02:16 am (UTC)(link)
I'm not asking for creativity to be capped? I can't go through pages and pages of prompts and fills to see if something catches my eye, to fill one out and then it gets lost in the comments. If the mods don't have a tagging system to find or fill comments, things get lost.

Creative caps are different than quantity caps imo, especially if things are going to be based on points. If this wasn't a point based competition, I wouldn't mind. But since it is, there should be something in place to gauge or handle how points are dealt out.

If someone can push out a well thought out 1k fic/full illustration and get the same amount of points for a 400 word fic(since its word count based)/rough sketch, that isn't fair. It discourages from creating quality content to give to someone who will enjoy it and not get short changed.

If bonus rounds are to just create ANY and ALL type of content, then why have a point system for it and not just a "x members made y amount of fills so this many number of points to team y" that way quality and quantity aren't relevant. Just the number of fills they did. im sure there's a better way to articulate this, which is why i didn't elaborate more on the comments and fills because it's a hard thing to judge on what is and isn't worth full points.
Edited (edit bc 400 was the min word count whoops) 2016-08-28 02:24 (UTC)
catlarks: (SASO: Cards)

(frozen comment)

[personal profile] catlarks 2016-08-28 02:35 am (UTC)(link)
I think we're using "creativity" differently. It sounds like you mean "imagination" while I mean "creation" as in "production." So what you're saying is a cap on creation, that's what a cap on quantity means. I don't think there should be a quantity limit.

We're going to fundamentally disagree because as soon as you say "what's worth points" you lose me. Not everyone has the same skill level, and I really love that SASO is an event that makes itself accessible to people who might be new to writing fic or drawing fanart, as well as to "seasoned veterans." I find the idea of arbitrating quality and telling some people they "just weren't good enough" to be given points when what looks phoned in to you might genuinely be the best they can do to be elitist and really disheartening.

IMO the points exist as motivation, and not a lot more. You get literally nothing for winning the event. To place more value on the points than accessibility and fun with the event is, imo, a mistake.
catzuya: (Default)

(frozen comment)

[personal profile] catzuya 2016-08-28 03:33 am (UTC)(link)
thanks for being civil with this, tbh.
like i said before, im p sure someone else can properly articulate on how to better the BR comments/fills. its feedback and good and bad should be seen. If the mods make something better or choose not to do anything, that is their choice completely. it's just a suggestion that i made, nothing more or less.

I know i can't eloquently word out what my brain is trying to say, and this is certainly not an attack on the fillers who worked hard to meet the max goals.

This is why i didn't touch or elaborate on it at all in my original post. As an artist, i know that skill levels vary, as well as skills in writing (writing is hard too, im amazed at how many words got pumped out gj guys)

a tldr of what i meant originally is:
prompter has a prompt> did it meet the prompt? y/n? y it gets tallied, n it doesn't.

but yes, agree to disagree, this is something for mods to make a final decision on if its an actual issue, if not, i'm not going to question it further.

catlarks: (SASO: Cards)

(frozen comment)

[personal profile] catlarks 2016-08-28 04:17 am (UTC)(link)
Likewise, honestly.

Some people are getting very heated in their opinions, it feels like, and while these are things I feel strongly about I think it's also really important to express oneself calmly and thoughtfully. Easier to make sense of suggestions and feedback that way too, tbh.

IMO "did it meet the prompt" is too subjective to be enforceable. The benefit of the current system is that a computer program CAN verify a word count, or an image size, and what is required of the mods is overseeing the program and checking tags. In an event producing hundreds upon hundreds of works a round, hand-checking something subjective (versus a yes/no question like "is this really 400 words" or "did a character die and the poster didn't include a major character death tag") requires a number of man-hours which is frankly OBSCENE to ask of volunteers.

And I'm aware one response to this issue is "hard caps, then it's less to check" but the mods have done that in the past. The mods have hand-checked fills in the past.

In the end it is a subject mods will make a final decision on, I just figure you might like to know that some of these things were in effect in previous versions of the event and worked VERY BADLY, and so these things were taken out for that reason.

TBH the thing I'd most like to see with BR fills isn't less of them, but some sort of indexing. I think it's a lot to ask of the mods but Karahashi single-handedly maintained an index for BR2 and BR3. Honestly, I think it'd be really great to crowdsource that same sort of resource, like big kink memes used to do with delicious. (did you ever participate in any of the bigger kink memes? Some of the organization was HARDCORE but that's because the mods outsourced to the users, to LOTS of users.)
catzuya: (Default)

(frozen comment) (If I go off tangent it's not on purpose I swear but mobile isn't kind to replies)

[personal profile] catzuya 2016-08-28 08:38 am (UTC)(link)
Mmm, I think people are getting upset because others are essentially commenting saying an OP feedback isn't right/valid.
Everyone is getting on the defensive for things that are meant for mods to make a final decision on, we can suggest all we want but we aren't mods so friendly talk and discussion should be encouraged, not calling out people or name calling.

I'm always a fan of constructive critisim and how things can be improved. Communication is key!!

This is people's first time participating, so the history isn't known very well for newbies. Me included. so knowing certain things like what did and didn't work for previous rounds/events would help when handling this sensitive topic and would make me rethink or think of another suggestion. Or if that suggestion had already been done and didn't work for some reason.

Indexing and tagging fills would be wonderful. Again I'm on mobile 99% of the time and clicking and filtering are my saviors. I'd honestly love a delicious type archive, but of that didn't work, I'd love a comm for BR. Tags can be used to filter things out, entries are contained along with their fills, and the subject line can be used better like kink memes used to do. (I vaguely looked around the fills for kink memes mostly BC I draw not write LOL. But agreeded the organization was top-notch.
catlarks: (SASO: Cards)

(frozen comment)

[personal profile] catlarks 2016-08-28 08:46 am (UTC)(link)
Yeah definitely! I really like the talkback post as a discussion forum for the userbase as well as a resource for the mods because besides the complaints, people have a lot of nice things to say, and share stories about their experience? Those things are so nice to read.

Tbh I worried a little after the fact that I came off kinda jerkish for referencing stuff from the previous years of the event the way I did. Like... I def don't mean to sound condescending with it, it's like you say: this is a first time for a lot of people so they don't know the history. I want to be able to give info about previous runs of the event, that's all.

(like, man oh man if you ever want horror stories, I can tell you Some Stuff about the early years. Some of my stories are really fun and nostalgic, too? I'm really fond of SASO/HSWC at this point because this is my fourth year so it's just "this kinda crazy thing I do every summer.")

I've been kinda poking the other people talking about putting together an index or a cross-post location because this is something I did get a response from the mods on: if the users want to run this resource, the mods will sanction it and promote it. So I genuinely really want to see who's interested in maintaining an index, and what format they think will be the best tradeoff of "easy to maintain" versus "easy to access." (having something mobile-accessible, as you mention, would be pretty great.)
catzuya: (Default)

(frozen comment)

[personal profile] catzuya 2016-08-28 09:06 am (UTC)(link)
More minds brainstorming, more common bugs get seen, that's the beauty of it. If it keeps getting brought up, then obvs there is a problem that needs to be addressed. (If it can't be fixed, explain why, if it can, try.)

Mm, it wasn't too bad honestly? Knowing you had previous experience just tells me you do know why x didn't work and that hs more pull over what I think might work.
Veterans do know what they're talking about.
Elite as it may sound, you guys have experience to back your claims, and that's important in things like this where a mod cannot be around to put out fires. Giving feedback with your name attatched is stressful for some newbies, so knowing a vet can come in and say "oh man one year we did that and it backfired terribly :( it is better though!" reassures me that this has been tried before with bad results.

Maintaining an index Would be good for people trying to find prompts, I saw that was a concern for smaller fandoms trying to fill for something they liked. Accessibility is my main concern for those things, Maintance can be scheduled and divided if you have multiple people working on this, it may not be a current list, but it will be something and that's better than nothing.
catlarks: (SASO: Cards)

(frozen comment)

[personal profile] catlarks 2016-08-28 09:26 am (UTC)(link)
Yesssss, exactly that. It's honestly fascinating to me that like, as a group we get repetitive with certain topics but this just serves to form a consensus? Some of the repeating is riffing on other people's ideas with new ones, some of it is multiple people autonomously suggesting the same thing, and tbh it's really cool just on a "social group" level.

Good to know! I feel like if you do it wrong "pulling rank" about being a veteran and having seen more years of an event can make people feel really bad, but it's also just a useful knowledge base? I've always liked being able to answer friends' little questions about the event so they don't need to bug a mod; I've ingested so many of the events rules to the point of being able to repeat them on command, ahaha.

(I think voting in particular, and MAIN ROUNDS in particular, have undergone the most obvious big changes over the years. Like, in 2013 there were technically four main rounds! It was honestly kind of horrible! Two actually feels so reasonable by comparison.)

I want an index So Bad, it's useful on so many levels. I love the thought of being able to sort for prompts based on several variables, and for fills by that same token. I honestly feel like the biggest drawback of SASO is just being able to FIND all the content for your ship or fandom. (and maintenance does require a large enough crew to split it up into manageable chunks. I sincerely DON'T think it's something mods should focus on, but like. We could do it. We could make a really nice reference thing.)

(in the same vein of old stories: a friend maintained a prompt index in 2014 cross-referenced by character and then ship and it was honestly super thorough and really beautiful, but it took her sooooo much time. Splitting the load is... So helpful...)

I feel like we've halfway wandered off into just chatting and tbh it's sort of nice but here I still am, on the DW post. This is fine.
clefairy: (Default)

(frozen comment)

[personal profile] clefairy 2016-08-28 02:54 pm (UTC)(link)
I didn't see this until after I posted my longass tl;dr but yes, I was of the opinion that a crowdsourced-to-participants index for prompts could probably be pretty easily maintained, even just using googleforms/sheets with options to sort tbh.

also omg I didn't know that MR votes were public last year until I read this feedback thread and saw it brought up multiple times and went back to check voting posts from last year and...that's...hORRIFIC.........
catlarks: (SASO: Cards)

(frozen comment)

[personal profile] catlarks 2016-08-28 05:01 pm (UTC)(link)
Yep, that's how it was last year! I assume a lot of people just don't have context for how voting has evolved, because there isn't really a space in the rules where you can go "and this is what it was like before, and it sucked."

Previously, the mods used DW polls because it was the only option that would verify someone was ACTUALLY signed up for the event; people have suggested other pre-existing options, and that's the problem with them. The voting program used this year was coded specifically for SASO and it just wasn't possible to test it on the scale of the event ahead of time, because where would you even get 300 people to test your program before the event starts? I know I actually think it's REALLY COOL how well the code worked, downtime aside, for this being its ~maiden voyage~

(and I for one very much prefer the opacity of this voting system, versus being able to see the results unfolding live and being able to change your vote in accordance with trends.)
clefairy: (Default)

(frozen comment)

[personal profile] clefairy 2016-08-29 05:30 am (UTC)(link)
Yoo the entire idea that you could change your votes to vote strategically sounds absolutely terrible. If it was going to be a transparent poll then edits shouldn't have been a thing, but that's all a bygone.

The opaqueness is definitely a much better thing right now, though I personally am curious of the final tallies after results are announced just to see how big the gaps were (or how close they were). Though that might just be me and some prefer to live in ignorance.