Yesterday (05.05.10) I (dsp) was reminded that we don't have many benchmarks for smell detection available. Here is what I'm currently aware of. All the samples contain bad smells although they are not annotated. … and these projects are small. Still it's worth to make a beginning .

• Marting Fowler
  • The video store example from his seminal Refactoring book. (The code can be found here)
  • He elaborated a longer example that got rejected for the book, but is of course even more interesting as a benchmark for us. (Code¹⁾)
• William C. Wake
  • Short articles discussing refactoring examples
  • Refactoring Workbook
• The LAN-simulation: A Refactoring Teaching Example
  • Serge Demeyer, Filip Van Rysselberghe, Tudor Girba, Jacek Ratzinger, Radu Marinescu, Tom Mens, Bart Du Bois, Dirk Janssens, Stephane Ducasse, Michele Lanza, Matthias Rieger, Harald Gall, Mohammad El-Ramly²⁾: The LAN-simulation: A Refactoring Teaching Example. Principles of Software Evolution, International Workshop on, pp. 123-134, Eighth International Workshop on Principles of Software Evolution (IWPSE'05), 2005.
  • http://lore.ua.ac.be/Research/Artefacts/refactoringLabSession/
  • Smell: Code duplication → Refactoring: Extract method
  • Smell: Feature envy → Refactoring: Move method
• The "Three Key Cup App" for Refactoring exercises javawarmup.threekeycupapp.zip
  • Smell: Feature envy → Refactoring: Move method
  • Smell: Code duplication → Refactoring: (here) Tease apart Inheritance, Introduce State Pattern
  • Smell: Dependency cycle → Refactoring: Introduce Observer Pattern
• Refactoring exercise from our Object Oriented Software Construction lecture, “Rectangle”
  • Task and code
  • Smell: Feature envy → Extract + move method
  • Smell: Comments (in method body telling the “what” not the “why”) → Extract method
  • Smell: Code duplication → Extract method
  • Smell: Dependency cycle → Extract Interface & Use Super type where possible
  • "Benchmark" results:
• Cohesion-tests - A collection of Java test classes for use in evaluating object-oriented cohesion metrics.
  • Desription: “This project provides a collection of Java test classes for use in evaluating object-oriented cohesion metrics. The classes vary in the amounts of connectivity within the classes' members, and in the types of connectivity. These classes should also be useful in evaluating the effectiveness of automated Extract Class refactoring tools.”
  • Examples developed/collected by Keith Cassell.
  • Not yet explored further by us.
• Premature Over- Under-design
  • Not so few Java programmers tend to create artificial complexity³⁾ in their code. Unfortunately removing this complexity seems to be even harder than removing the standard smells.
  • William Woody gives an excellent idea about how this kind of code feels in his blog post "How (not) to write Factorial in Java."
  • My (dsp) candidate name for the smells that would detect this artifical complexity is “Occams razor set”.
• Further Candidates (Not explored further)
  • http://www.computer.org/portal/web/csdl/doi/10.1109/WCRE.2009.23 - Reports about God Classes in Open Source projects.
  • http://101companies.org/ - A project to compare the implementation of one application in different languages.
  • http://sourcerer.ics.uci.edu/index.html - Aimed at exploring the open source phenomenon through the use of code analysis.
  • http://qualitascorpus.com/ - A curated collection of software systems intended to be used for empirical studies of code artefacts
  • http://www.moosetechnology.org/docs/pangea - Was created as a complement to the Qualitas Corpus project.
  • http://java.labsoft.dcc.ufmg.br/qualitas.class/ - Compiled version of the Qualitas Corpus.
  • http://www.ptidej.net/tools/designsmells/index_html#2 - Anti-Pattern - Micro-Architecture Repository, database of occurrences of code and design smells.
• Focused on design patterns, might nevertheless be useful for reference:
  • http://www.ptidej.net/tools/designpatterns/index_html#2 - Pattern-like Micro-Architecture Repository
  • http://essere.disco.unimib.it/DPBWeb/ - Design Pattern detection tools Benchmark platform