SE & SDA Wiki

Software Engineering for Smart Data Analytics & Smart Data Analytics for Software Engineering

User Tools

Site Tools


Test Framework

Usage

There exists one test project for each cultivate project. Additionally you need to checkout the cultivate.testframework platform project , which contains common files for all tests.

The test project should be named as the project to be tested e.g. cultivate.basic.test corresponds to cultivate.basic. The testframework will try to consult /“prolog/load.pl” in the project to be tested, but will also give an error if this file does not exist.

The following directories may exist in each test project:

  • plain-tests: Contains plain JUnit Java tests.
  • plugin-tests: Contains JUnit tests that need an eclipse environment to run.
  • prolog-tests: Contains tests for prolog predicates, also based on JUnit.

For writing prolog tests take a look at the demo.APITest example class in cultivate.basic.test:

Creation of a new test consists of 4 steps:

  1. Write the test data
  2. Generate the factbases
  3. Write the test
  4. Run the test

First you need to write your testdata. Do this by creating a new testdata-* src folder and adding it to the build path. Now write your packages and classes. Afterwards you must run the generator-factbases-project.launch via the eclipse launcher menu. This will create the file *.qlf in the factbases directory. Now you can write your test by extending the AbstractPrologTestCase. It will automatically consult the prolog file prolog/load.pl of your project.

The framework supports hamcrest matcher, which should be used. See the Class APITest.java in *cultivate.basic.test/prolog-test/demo/APITest.java*

API

  • List<String> resultOf(String functor, String… arguments):

uses queryOnce and gives only the unifications for one call of the predicate.

  • List< List<String> > allResultsOf(String functor,String… arguments):

uses queryAll, gives all results of the predicate through backtracking

  • String idOf(String FullQualifiedName):

returns a PEF Id for the given full qualified name, see the prolog documentation of id_of_named_element/3 for details.

  • boolean call(String predicate):

calls the predicate without giving results (only success), useful for calling a generation predicate.

  • consult(String filename): Consults additional prolog files given by filename.
  • consultResource(String filename): Consult prolog file in the current source-folder's resources folder
  • useTestData(String testdataName): Consults the factbase given by factbaseName. Note: If the directory is called testdata-123 you only have to give 123 as testdataName.
  • consultJTFacts(): Will consult the JTransformer facts, test if this is needed first (Tests will run slower).
  • metric(String Metricname) and metric(String Metricname, String elementID): both are convenience method for testing metric values, this method uses allResultsOf.

Matcher

The basic Matcher are contained in the CultivateMatcher and the Hamcrest Matchers factory classes.

  • succeeds(): used for testing if the query should succeed
  • fails(): used for testing if the query should fail
  • hasAnswers(int n): Test for a specific size of the list, only useful for allResultsOf(..)
  • hasItems(String… elements): Tests for containment in the resultOf result.
  • has(values(Object… values),…): Tests for containment in the allResultsOf result. values(..) corresponds to one call result of the predicate, the order of the values(..) is irrelevant.
  • hasValue(Object testvalue) and hasValues(Object.. testvalue) are both convenience methods for has(values(testvalue)).
  • both(Matcher).and(Matcher) or either(Matcher).or(Matcher) can be used to logically connect Matchers
  • Every possible Hamcrest Matcher is usable.

A tour of common Hamcrest Matchers

The following list is taken from the official hamcrest website (May 2011).

  • Core
    • anything - always matches, useful if you don't care what the object under test is
    • describedAs - decorator to adding custom failure description
    • is - decorator to improve readability - see “Sugar”, below
  • Logical
    • allOf - matches if all matchers match, short circuits (like Java &&)
    • anyOf - matches if any matchers match, short circuits (like Java ||)
    • not - matches if the wrapped matcher doesn't match and vice versa
  • Object
    • equalTo - test object equality using Object.equals
    • hasToString - test Object.toString
    • instanceOf, isCompatibleType - test type
    • notNullValue, nullValue - test for null
    • sameInstance - test object identity
  • Beans
    • hasProperty - test JavaBeans properties
  • Collections
    • array - test an array's elements against an array of matchers
    • hasEntry, hasKey, hasValue - test a map contains an entry, key or value
    • hasItem, hasItems - test a collection contains elements
    • hasItemInArray - test an array contains an element
  • Number
    • closeTo - test floating point values are close to a given value
    • greaterThan, greaterThanOrEqualTo, lessThan, lessThanOrEqualTo - test ordering
  • Text
    • equalToIgnoringCase - test string equality ignoring case
    • equalToIgnoringWhiteSpace - test string equality ignoring differences in runs of whitespace
    • containsString, endsWith, startsWith - test string matching

Old API

The following methods are deprecated and should not be used.

Additionally you have the following helper methods to use:

  • callPredicate(String query): This will call a predicate and return all results as List.
  • assertResultSize(int size): Assert the number of results of the last called predicate.
  • setExpectedResults(String key, String… values): Sets the expected results for the given key. Can be called more than once with different keys. Tuples of values from each call will match then.
  • assertExpectedResultsExist: Asserts the expected result tuples have been returned by the predicate call.
  • assertPredicateReturnsFalse: Asserts that the predicate has an empty result set
  • assertPredicateReturnsTrue: Asserts that the predicate has a non-empty result set
  • getIdOfFullQualifiedName: Returns the ID for a qualified name
  • fullFilename: Get the full filename of a (class)file in the sourcefolder
  • getIdOfSourceLocation: Get the ID of an element described by the the full filename and the location of the element
  • dumpResults: Writes a list to STDOUT of results of the last prolog query
  • dumpExpectedResultSettings: Same as dumpResults with other output format

For executing predicates in a changing context, the ContextualPredicate class is provided. it supports the following methods:

  • get/setContextName: to set the name of the context parameter in the predicate, defaults to Context
  • setVariablesToCheck: Defines the names of the variables to verify
  • assertResult: Asserts a result in a specific context. Note: If you execute several asserts in the same context, execute them in a row for speed-up.
  • assertResultSize(String Context, int size): Assert the number of results of the predicate in the Context.
  • dumpResult: see above
  • dumpExpectedResultSettings: see above

History

research/cultivate/test_framework.txt · Last modified: 2018/05/09 01:59 (external edit)

SEWiki, © 2018