SE & SDA Wiki

Software Engineering for Smart Data Analytics & Smart Data Analytics for Software Engineering

User Tools

Site Tools

Test Framework


There exists one test project for each cultivate project. Additionally you need to checkout the cultivate.testframework platform project , which contains common files for all tests.

The test project should be named as the project to be tested e.g. cultivate.basic.test corresponds to cultivate.basic. The testframework will try to consult /“prolog/” in the project to be tested, but will also give an error if this file does not exist.

The following directories may exist in each test project:

  • plain-tests: Contains plain JUnit Java tests.
  • plugin-tests: Contains JUnit tests that need an eclipse environment to run.
  • prolog-tests: Contains tests for prolog predicates, also based on JUnit.

For writing prolog tests take a look at the demo.APITest example class in cultivate.basic.test:

Creation of a new test consists of 4 steps:

  1. Write the test data
  2. Generate the factbases
  3. Write the test
  4. Run the test

First you need to write your testdata. Do this by creating a new testdata-* src folder and adding it to the build path. Now write your packages and classes. Afterwards you must run the generator-factbases-project.launch via the eclipse launcher menu. This will create the file *.qlf in the factbases directory. Now you can write your test by extending the AbstractPrologTestCase. It will automatically consult the prolog file prolog/ of your project.

The framework supports hamcrest matcher, which should be used. See the Class in *cultivate.basic.test/prolog-test/demo/*


  • List<String> resultOf(String functor, String… arguments):

uses queryOnce and gives only the unifications for one call of the predicate.

  • List< List<String> > allResultsOf(String functor,String… arguments):

uses queryAll, gives all results of the predicate through backtracking

  • String idOf(String FullQualifiedName):

returns a PEF Id for the given full qualified name, see the prolog documentation of id_of_named_element/3 for details.

  • boolean call(String predicate):

calls the predicate without giving results (only success), useful for calling a generation predicate.

  • consult(String filename): Consults additional prolog files given by filename.
  • consultResource(String filename): Consult prolog file in the current source-folder's resources folder
  • useTestData(String testdataName): Consults the factbase given by factbaseName. Note: If the directory is called testdata-123 you only have to give 123 as testdataName.
  • consultJTFacts(): Will consult the JTransformer facts, test if this is needed first (Tests will run slower).
  • metric(String Metricname) and metric(String Metricname, String elementID): both are convenience method for testing metric values, this method uses allResultsOf.


The basic Matcher are contained in the CultivateMatcher and the Hamcrest Matchers factory classes.

  • succeeds(): used for testing if the query should succeed
  • fails(): used for testing if the query should fail
  • hasAnswers(int n): Test for a specific size of the list, only useful for allResultsOf(..)
  • hasItems(String… elements): Tests for containment in the resultOf result.
  • has(values(Object… values),…): Tests for containment in the allResultsOf result. values(..) corresponds to one call result of the predicate, the order of the values(..) is irrelevant.
  • hasValue(Object testvalue) and hasValues(Object.. testvalue) are both convenience methods for has(values(testvalue)).
  • both(Matcher).and(Matcher) or either(Matcher).or(Matcher) can be used to logically connect Matchers
  • Every possible Hamcrest Matcher is usable.

A tour of common Hamcrest Matchers

The following list is taken from the official hamcrest website (May 2011).

  • Core
    • anything - always matches, useful if you don't care what the object under test is
    • describedAs - decorator to adding custom failure description
    • is - decorator to improve readability - see “Sugar”, below
  • Logical
    • allOf - matches if all matchers match, short circuits (like Java &&)
    • anyOf - matches if any matchers match, short circuits (like Java ||)
    • not - matches if the wrapped matcher doesn't match and vice versa
  • Object
    • equalTo - test object equality using Object.equals
    • hasToString - test Object.toString
    • instanceOf, isCompatibleType - test type
    • notNullValue, nullValue - test for null
    • sameInstance - test object identity
  • Beans
    • hasProperty - test JavaBeans properties
  • Collections
    • array - test an array's elements against an array of matchers
    • hasEntry, hasKey, hasValue - test a map contains an entry, key or value
    • hasItem, hasItems - test a collection contains elements
    • hasItemInArray - test an array contains an element
  • Number
    • closeTo - test floating point values are close to a given value
    • greaterThan, greaterThanOrEqualTo, lessThan, lessThanOrEqualTo - test ordering
  • Text
    • equalToIgnoringCase - test string equality ignoring case
    • equalToIgnoringWhiteSpace - test string equality ignoring differences in runs of whitespace
    • containsString, endsWith, startsWith - test string matching


The following methods are deprecated and should not be used.

Additionally you have the following helper methods to use:

  • callPredicate(String query): This will call a predicate and return all results as List.
  • assertResultSize(int size): Assert the number of results of the last called predicate.
  • setExpectedResults(String key, String… values): Sets the expected results for the given key. Can be called more than once with different keys. Tuples of values from each call will match then.
  • assertExpectedResultsExist: Asserts the expected result tuples have been returned by the predicate call.
  • assertPredicateReturnsFalse: Asserts that the predicate has an empty result set
  • assertPredicateReturnsTrue: Asserts that the predicate has a non-empty result set
  • getIdOfFullQualifiedName: Returns the ID for a qualified name
  • fullFilename: Get the full filename of a (class)file in the sourcefolder
  • getIdOfSourceLocation: Get the ID of an element described by the the full filename and the location of the element
  • dumpResults: Writes a list to STDOUT of results of the last prolog query
  • dumpExpectedResultSettings: Same as dumpResults with other output format

For executing predicates in a changing context, the ContextualPredicate class is provided. it supports the following methods:

  • get/setContextName: to set the name of the context parameter in the predicate, defaults to Context
  • setVariablesToCheck: Defines the names of the variables to verify
  • assertResult: Asserts a result in a specific context. Note: If you execute several asserts in the same context, execute them in a row for speed-up.
  • assertResultSize(String Context, int size): Assert the number of results of the predicate in the Context.
  • dumpResult: see above
  • dumpExpectedResultSettings: see above


research/cultivate/test_framework.txt · Last modified: 2018/05/09 01:59 (external edit)

SEWiki, © 2018