Hey all, OG SourceCred shill (aka founder) here. Thanks @METADREAMER for posting this. I’m seeing some great questions and pushback here and wanted to share my perspective.
On Gaming
IMO, the biggest mistake people make with gaming is treating it as a binary–as though a system is either vulnerable to gaming or 100% impervious. In reality it’s a continuum. As an example, look at Google. It definitely gets gamed through link spam and SEO, but is sufficiently robust that the system is still useful, even if bad actors get the advantage here and there. Another example is democratic politics: democracies get gamed in all kinds of ways (gerrymandering, lobbyists, etc), but it’s still the best system of governance that we’ve tried at scale.
The goal when designing social incentive systems then, is not to aim for a 100% perfect system, but one where the results are high enough quality to be useful.
Based on experience, I’m confident that SourceCred meets this quality threshold. In our own ~year of dogfooding, we’ve distributed about $700k through the algorithm, and haven’t had any serious issues with gaming. More recently, MakerDAO has distributed about $20k in their pilot usage and also haven’t had issues with gaming.
The robustness comes from a few sources. One is that, as @blacklotus mentioned, the PageRank algorithm is reasonably hard to game, and has some nice automatic properties of Sibyl resistance. The second is that SourceCred is open-source and totally configurable (both through changing the weights, and through changing the underlying data source). This means that if it starts getting gamed, the community can come up with new weights to penalize the gaming behavior. This really means that someone trying to game the scores is really trying to game the community. Communities of smart people are hard to fool.
As an example of the kind of configurability we have in mind: the MakerDAO community was worried that there could be issues around like brigading, especially since they sometimes see a bunch of new accounts being made to upvote new collateral proposals. So at their suggestion, we modified the heuristics to take Discourse trust levels into account. In the long run, I think we’ll treat gaming vulnerabilities in SC like security vulnerabilities in software in general: have bug bounties, responsible disclosure, and have being open-source be a source of robustness.
On Monetary Rewards & Sustainability
I think there’s a solid case to be made that the community doesn’t need any monetary incentives for governance at this point. People in the community are super passionate, and many of them already have plenty of YFI, so they’re already aligned. However: will this be sustainable in 2-5 years? Some of the early adopters will have moved on or lost interest, and lots of the new faces won’t have much YFI. Meanwhile the governance overhead will only grow as YFI becomes more systemically important. This something Maker ran into –the governance overhead kept getting bigger, and relying on volunteer labor became unsustainable.
That said, I think creating a super “gamified” contribution experience would be a big mistake. It would orient people too much on extrinsic rewards. SourceCred is actually kind of designed as anti-gamified contributions. There’s no fast loop of make post, get like, get money. Instead, the algorithm is always looking retroactively at the long-run value of your contributions, and rewarding you based on the totality. The idea is less to cultivate a mindset of “I need to get 5 likes today to meet my quota” and more “If I make substantial long run contributions, I’ll get a solid long-run reward”.
Closing Thoughts
I think y’all are really right to be skeptical of using a system like this. The potential downsides of incorporating a broken incentive system are huge, so it pays (pun unintended) to be cautious. For my part, SourceCred is my life and passion project, and understanding it as well as I do I was still really worried about activating it for SourceCred. What if it wrecked the community and shifted everyone to fighting about Cred all the time?
So I started really small–in the first week we only distributed $500. We’re now at a point where we distribute $25k per week, but we built the confidence to do so over many months, and as we all learned how to use the tool. You can also take a look at what the Maker community’s initial thoughts were , and similarly they started really small–about $1k in the first month–and have ramped up usage as they’ve gained confidence. Just using it as a leaderboard for a while to see how it behaves is a very reasonable approach.
Ultimately, SourceCred’s goal is to enable truly decentralized communities like YFI. Because it’s hard to directly value contributions, and thus to have decentralized rewards, projects often wind up forced to set up centralized foundations to hire employees, and we’re right back in the old world. The goal of SC is to make decentralized communities (and open-source more broadly) sustainable in the long run. So, I’m hopeful that you’ll find it useful. But keep rocking on either way. 