§ Advances to competitive Pokémon

Shoddy Battle was released by Cathy and bearzly on July 7, 2007 and it ushered in a number of significant advances in the Pokémon community, changing the face of competitive Pokémon in ways that have been continued by subsequent simulators. This page will describe some of Shoddy Battle's contributions to the competitive Pokémon community.

Table of contents:

You can click on the § symbols to return to the table of contents.

On another page, you can also read about the history of Shoddy Battle.

§ Free, open source, and cross-platform software

Shoddy Battle was and is free and open source software, which means that the source code was and is available for users to review, learn from, modify, and redistribute pursuant to the applicable free software licence.

In July 2007, a free and open source Pokémon simulator was a fairly revolutionary idea. NetBattle, which was the main simulator used for many years before Shoddy Battle, was closed source and this led to a number of problems, including the following:

Shoddy Battle was free and open source from the start. Users were encouraged to inspect the source code and learn from it. Even users who did not contribute were able to use the source code to better understand the Pokémon game mechanics or to detect errors. Additionally, Shoddy Battle was initially coded almost entirely in Java and was able to run on all common operating systems including Microsoft® Windows®, Apple® Mac OS X®, and GNU/Linux.

The fact that Shoddy Battle was free and open source led to a variety of long-lasting positive effects:

The open source nature of Shoddy Battle was so beneficial for the Pokémon community that all subsequent Pokémon simulators have been free and open source. In fact, within weeks of Shoddy Battle's release in July 2007, the developers of upcoming rival simulators, such as Smogon Competitor, announced that their formerly closed source simulators would instead be open source. As of December 2012, the current most popular simulator, Pokémon Showdown, is also free and open source.

§ Scalability

The 4th generation of Pokémon games began with the release of Pokémon Diamond and Pearl in September 2006 in Japan and April 2007 in North America. Prior to the 4th generation, online competitive Pokémon was a relatively niche activity. However, the release of the 4th generation games brought unprecedented levels of traffic into the fold of online competitive Pokémon.

Shoddy Battle was prepared to handle the rapid increase in traffic. It supported hundreds of concurrent users (there were typically around 500 users online at a time for most of its life) and it was able to host hundreds of thousands of battles per month. Its high scalability meant it was able to handle ever-increasing traffic, far above what was initially contemplated when the program was written and far above what previous simulators were able to handle.

The following graph shows the number of battles played per month on the Official or Smogon server (the most popular server) between October 2007 and July 2010, inclusive:

Battles played per month on the Official/Smogon server (view data)
Battles per month

As can be seen from the graph, the amount of traffic increased rapidly to reach over six times the initial levels (which is the level of traffic that was that contemplated during development). At its peak, around 700,000 battles were played per month on the Smogon server — around one battle starting every four seconds. Shoddy Battle served the community admirably by handling these high levels of traffic. Between October 2007 and July 2010, there were around 12.3 million battles played on the Official/Smogon server.

It is also worth noting that the Official/Smogon server was not the only server. There were a number of other large servers, such as the Create-a-Pokémon Project server and the Pokemonexperte server, as well as a variety of smaller niche servers.

In the years following Shoddy Battle, the popularity of online competitive Pokémon has continued to grow and subsequent simulators have had to continue the tradition of high scalability set by Shoddy Battle.

§ Extension and customisation

Unlike previous simulators, Shoddy Battle was designed from the ground up to be extensible and customisable by users.

Modification of the game

In Shoddy Battle, the individual moves, abilities, items, rules (called "clauses"), etc. were implemented using a flexible and intuitive API in order to facilitate modification, extension, and the creation of new moves, etc. The framework also allowed for the creation of entirely new Pokémon characters. Some changes could be made with no programming, and even when programming was required, it was simple and within reach of non-programmers. Many users created modified Pokémon games. The most popular sub-community centred on modifying on the game was the Create-a-Pokémon Project, but there were other groups and individuals as well.

The client-server protocol was designed so that the standard client could be used to play on a server running a modified Pokémon game. This allowed people who modified the program to share their work without their users having to download any additional software. The standard Team Builder allowed users to build teams designed to work on any modified Pokémon game.

Modularity

Shoddy Battle was designed in a modular fashion so that one part of the program could be used without the others. For example, several other projects used Shoddy Battle's implementation of the Pokémon game mechanics, including PokeNet Global, a now-defunct MMORPG.

The modular nature of Shoddy Battle meant that it was relatively easy to write tools that interoperate with the software. For example, Martin wrote a tool to import teams from a plain text format. Cathy wrote a tool to generate teams based on the information in the Smogon Strategy Pokédex, a community-maintained repository of information on Pokémon game strategy. (Unfortunately, this tool is no longer available as of December 2012.) Users released a number of interesting tools over the years.

Bots

In contrast to previous simulators, the network protocols for Shoddy Battle and Pokémon Lab were well-documented (and the source code was available) and simple to use. In addition, Shoddy Battle and Pokémon Lab came with a bot framework that allowed users to readily write bots in any one of several languages (Java, Python, etc.) to join the server and interact with users.

Over the years, users wrote bots to do the following sorts of tasks:

One impressive bot is Technical Machine, an AI project by david stone that plays intelligent Pokémon matches on Pokémon Lab (and some other simulators) using a minimax-based algorithm.

Outlook

Shoddy Battle set the bar high for extensibility and customisation features. Later simulators have provided similar facilities in order to satisfy user need; for example, Pokémon Showdown also supports modified Pokémon games.

§ Statistical data and user ranking

Shoddy Battle was the beginning of a new era of Pokémon based heavily on statistical data.

Shoddy Battle was the first simulator to provide statistical information on what Pokémon and strategies were used by players. Before Shoddy Battle, there was no statistical data of this nature available. Shoddy Battle was also the first simulator to provide players with a rated matchmaking facility (the "ladder").

Shoddy Battle's focus on statistics had some significant and long-lasting effects on the community.

Matchmaking and user rating

Previous simulators had included basic mechanisms to keep track of how many matches a person had won or lost. However, Shoddy Battle was the first simulator to take a systematic and statistical approach to user rating. The Shoddy Battle matchmaking and rating system included the following features, all of which were novel for competitive Pokémon at the time.

Rating system

Rather than a simple count of wins and losses, Shoddy Battle kept track of player skill using the Glicko2 rating system. Rather than conceptualising a player's skill as a particular number, each player was assigned a probability distribution; a player's "played skill" realised in any particular game was understood as a random variable distributed according to a normal distribution characterised by two parameters: μ (mean) and σ (standard deviation).

The value of these parameters was adjusted based on game outcomes to reflect both the player's average "played skill" in a particular match (μ) as well as the level of uncertainty in that value (σ). For the purpose of ranking players, Shoddy Battle used a conservative estimate of the player's average "played skill" (defined as μ - 4 * σ). The value of σ also increased with player inactivity, reflecting an increased uncertainty in the average rating, but also serving to encourage people to play regularly, increasing activity on the server.

The sophisticated rating system provided for a more competitive environment and contributed to the popularity of the ladder system.

Selection of opponents

Shoddy Battle did not allow players to select their own opponents for rated matches. Rather, match pairings were made "randomly" based on the relative ratings of players.

Ideally, people would be matched to somebody of similar skill to them because the game outcome of such a pairing provides more information about the skill of each player than does a match between unequal opponents. In a match between unequal opponents, the result gives very little information because it is already known that one player is much better than the other and the outcome does not help determine the degree to which that is the case (without attempting to quantify "how badly" somebody lost, which has a variety of practical and theoretical flaws, and which Shoddy Battle did not attempt to do).

Aside from the rating-theoretic benefit of matching players "randomly" based on relative ratings, the system also produced matches that would hopefully be more fun for the players. In addition, the system prevented cheating by way of fixing match outcomes because a player would not be able to guarantee a match against the collaborator in such a scheme.

Uniformity of ruleset

In Shoddy Battle, rating statistics were "local" to a particular ruleset. In other words, for each ruleset, a player had a different set of rating statistics. The purpose of this was to avoid the effect that playing under different rules might have on a player's "played skill". In theory, we might imagine that there is some underlying notion of "skill" which is predictive of skill in any ruleset; however, measuring such an underlying notion of skill has many pitfalls and typically results in meaningless ratings. Accordingly, Shoddy Battle localised rating statistics to a particular ruleset.

This system also helped players find matches quickly because it reduced the variation in rulesets being played. In Pokémon, a team tends to be designed for a specific ruleset, and the ladder system made it much easier to find a match under the desired ruleset.

Type of Pokémon statistics collected and reported

Shoddy Battle collected statistics on what Pokémon and strategies players were using in their battles. The presence of the matchmaking and rating system meant that usage statistics for ladder games could be assumed to be representative of competitive play, because all players had an incentive to play to win.

At first, the statistics collected and reported were fairly basic and consisted of information on how often each Pokémon was used in each ruleset as a team-member and as a lead (initial Pokémon). Even these basic statistics were unprecedented in the competitive Pokémon community and had immediate practical applications and theoretical consequences (discussed below). As time went on, the statistics collected and reported became increasingly detailed and informative.

The statistics for Cathy's server from October 2007 to July 2008 are archived on this website.
The statistics for Smogon Server from July 2008 to July 2010 are archived on Smogon.

In addition to the basic statistics on the popularity of each species of Pokémon, the statistics collected and reported over the years included the following:

Various other Pokémon statistics were computed from these and reported for further analysis.

Statistics and tiering

One of the major applications of the statistics collected by Shoddy Battle (both the user rating statistics and the Pokémon statistics) was tiering. In the context of competitive Pokémon, "tiering" refers to the creation of different several rulesets by banning particular Pokémon from each ruleset.

Construction of a mode of play featuring less commonly seen Pokémon

One problem that tiering solves is the fact that there are hundreds of Pokémon, but most of them are not usable in the standard game because they are too weak to be effective. By removing the most commonly seen Pokémon ("OU" or Overused Pokémon) from the game, a different game is created ("UU" or Underused) which provides an alternative mode of play.

Prior to the advent of Pokémon statistics, the construction of tiers had been based on intuition rather than any sort of mathematical metric. The introduction of usage statistics allowed the construction of tiers based on actual usage data, so that the UU ruleset would actually feature less commonly seen Pokémon, which was the intent.

Banning of exceedingly strong Pokémon

A second function of tiering is to remove Pokémon that are deemed "too strong" for a particular ruleset. Pokémon removed from the standard game for this reason were referred to as "Ubers", whereas Pokémon removed from the UU game for this reason were referred to as "BL".

Shoddy Battle's statistics helped in the determination of which Pokémon were "too strong" in a couple ways:

Statistics and strategy

The statistics provided by Shoddy Battle also affected strategy in the Pokémon game itself.

One of the major skills involved in the game of Pokémon is "prediction"; that is, determining what the opponent is going to do based on imperfect and incomplete information. The existence of a treasure trove of statistical information about what people commonly do in a similar situation provides useful information to assist in predicting what one's opponent is likely to do in a particular battle. Of course, the opponent is also aware of the statistical data and as such may alter her strategy accordingly, which must be taken into account as well. David Sirlin refers to this phenomenon as Yomi Layers and it is a hallmark of complex competitive games.

Moving beyond human players, the statistical information also made it possible to write sophisticated Pokémon-playing AI bots whose decisions of what moves to make were informed by the Pokémon statistics. The most sophisticated AI bot to date is Technical Machine, which relies heavily on a variety of different Pokémon statistics in order to predict teams, evaluate game states, and so on.

Outlook

Shoddy Battle significantly changed the landscape of the competitive Pokémon community by providing player ratings and Pokémon statistics. Statistics have become a central aspect of the competitive Pokémon community and now figure prominently in all manner of competitive Pokémon discussions and in the formulation of strategy. All subsequent simulators have continued to provide statistical facilities, inspired by the features in Shoddy Battle.

§ Status

With the conclusion of the 4th generation, Shoddy Battle has been superseded by other simulators such as Pokémon Showdown. Shoddy Battle remains notable for the changes it effected in the Pokémon community. Later Pokémon simulators have followed the path set by Shoddy Battle and continue to expand on the above advances. On another page, you can also read about the history of Shoddy Battle.

If you enjoyed using Shoddy Battle or have questions or comments, please feel free to email Cathy and bearzly at developers@pokemonlab.com and let them know. They appreciate hearing from you. You can also visit Cathy's website at cathyjf.com.