SDA SE Wiki

Software Engineering for Smart Data Analytics & Smart Data Analytics for Software Engineering

User Tools

Site Tools


Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
teaching:labs:xp:2009a:test_framework [2009/03/26 08:18]
pascal
teaching:labs:xp:2009a:test_framework [2018/05/09 01:59] (current)
Line 1: Line 1:
 +====== Test Framework ======
 +
 +===== Usage =====
 +
 +There exists one test project for each cultivate project. Additionally you need to checkout the //​cultivate.test//​ project, which contains common files for all tests. The following directories may exist in each project:
 +
 +  * //​plain-tests//:​ Contains plain JUnit Java tests.
 +  * //​plugin-tests//:​ Contains JUnit tests that need an eclipse environment to run.
 +  * //​prolog-tests//:​ Contains tests for prolog predicates, also based on JUnit.
 +
 +For writing prolog tests take a look at the //​MisnamedTestCaseTest//​ example in //​cultivate.basic.test//:​
 +
 +{{:​teaching:​labs:​xp:​2009a:​testsample.jpg|}}
 +
 +Creation of a new test consists of 4 steps:
 +
 +  - Write the test data
 +  - Generate the factbases
 +  - Write the test
 +  - Run the test
 +
 +First you need to write your testdata. Do this by creating a new //​testdata-*//​ src folder and adding it to the build path. Now write your packages and classes. Afterwards you must run the //​generator-factbases-project.launch//​ via the eclipse launcher menu. This will create the file //*.qlf// in the //​factbases//​ directory. Now you can write your test by extending the //​AbstractPrologTestCase//​. It will automatically consult the prolog file //​prolog/​load.pl//​ of your project. Additionally you have the following helper methods to use:
 +
 +  * //​consult(String filename)//:​ Consults additional prolog files given by filename.
 +  * //​consultResource(String filename)//:​ Consult prolog file in the current source-folder'​s ''​resources''​ folder
 +  * //​useTestData(String testdataName)//:​ Consults the factbase given by factbaseName. Note: If the directory is called //​testdata-123//​ you only have to give //123// as testdataName.
 +  * //​consultJTFacts()//:​ Will cronsult the JTransformer facts
 +  * //​callPredicate(String query)//: This will call a predicate and return all results as List.
 +  * //​assertResultSize(int size)//: Assert the number of results of the last called predicate.
 +  * //​setExpectedResults(String key, String... values)//: Sets the expected results for the given key. Can be called more than once with different keys. Tuples of values from each call will match then.
 +  * //​assertExpectedResultsExist//:​ Asserts the expected result tuples have been returned by the predicate call.
 +  * //​assertPredicateReturnsFalse//:​ Asserts that the predicate has an empty result set
 +  * //​assertPredicateReturnsTrue//:​ Asserts that the predicate has a non-empty result set
 +
 +  * //​getIdOfFullQualifiedName//:​ Returns the ID for a qualified name
 +  * //​fullFilename//:​ Get the full filename of a (class)file in the sourcefolder
 +  * //​getIdOfSourceLocation//:​ Get the ID of an element described by the the full filename and the location of the element
 +
 +  * //​dumpResults//:​ Writes a list to STDOUT of results of the last prolog query
 +  * //​dumpExpectedResultSettings//:​ Same as //​dumpResults//​ with other output format
 +
 +For executing predicates in a changing context, the ''​ContextualPredicate''​ class is provided. it supports the following methods:
 +  * //​get/​setContextName//:​ to set the name of the context parameter in the predicate, defaults to ''​Context''​
 +  * //​setVariablesToCheck//:​ Defines the names of the variables to verify
 +  * //​assertResult//:​ Asserts a result in a specific context. Note: If you execute several asserts in the same context, execute them in a row for speed-up.
 +  * //​assertResultSize(String Context, int size)//: Assert the number of results of the predicate in the Context.
 +  * //​dumpResult//:​ see above
 +  * //​dumpExpectedResultSettings//:​ see above
 +
 +**IMPORTANT:​** Before you can run your tests with JUnit you have to alter //​cultivate.test/​src/​org.cs3.cultivate.test.framework/​test.properties//​ to match the correct paths.
 +
 +===== History =====
 +
 +==== What do we wan't? ====
 +
 +  * Stuff to test:
 +    - Java parts
 +    - Prolog parts
 +
 +  * General Criterias
 +    - Independence of tests and code
 +
 +  * Evaluation Criterias for Prolog tests:
 +    - Fast Tests
 +    - Automated Tests
 +    - Real classes as testdata
 +    - Clear relations between testdata, tests and code
 +    - Test cases should be easy to write
 +    - Test framework should be easy to use
 +
 +==== What do we have? ====
 +
 +  - Java Parts 8-)
 +    * Plain Tests: Test of java cultivate core functionality
 +    * Plugin Tests: Test of cultivate functionality related to eclipse
 +
 +  - Prolog Parts :-\
 +    * Prolog Tests (based on plugin tests):
 +      - Slow :-(
 +      - No automation because of a bug :-(
 +        * plwin does not close if test fails
 +      - Testdata are no real classes :-(
 +      - Testdata in the tests 8-)
 +      - Assertions not intuitive to check :-(
 +      - Easy to use, since junit test integration is good in eclipse 8-)
 +      - JTransformer predicates readily available for the tests. 8-)
 +      - Same internal mechanisms as in Cultivate itself. :-) 
 +    * Fitnesse Tests
 +      - Fast 8-)
 +      - Automation works 8-)
 +      - Real classes as test data 8-)
 +      - Traceability between testdata and factbases not given :-(
 +      - Easy writable tests through wiki syntax 8-)
 +      - Fitnesse is unintuitive for tests at the beginning :-(
 +      - JTransformer predicates need to be explicitly consulted. :-(
 +    * JTransformer (based on plugin-tests)
 +      - Slow :-\
 +      - Automation works 8-)
 +      - Testdata are real classes, but zipped :-\
 +      - Testdata in the tests 8-)
 +      - Assertions not intuitive to check :-(
 +      - Easy to use, since junit test integration is good in eclipse 8-)
 +
 +
 +==== What can we do? ====
 +
 +  * Assumptions:​
 +    * One project for the test framework (cultivate.tests)
 +    * One test project per project to test (e.g. cultivate.core => cultivate.core.tests)
 +
 +  * Fitnesse
 +    * One sourcefolder per factbase in each test project and precompile and filter factbases based on sourceFolderT 8-)
 +    * Prepare Fixture that consults all JTransformer predicates.
 +
 +  * Junit Tests
 +    - One sourcefolder per factbase in each test project
 +      * Still needs slow plugin tests :-/
 +    - additionaly precompile and filter factbases based on sourceFolderT
 +      * would get rid of plugin tests and move to plain prolog tests 8-)
 +    - Improve assertion checking for prolog results 8-)
 +    - Timeout for prolog tests, so that plwins get removed.
 +
 +  * Write a test environment for plain prolog tests
 +    * Java Prolog Queries not testable :-(
 +    * Relation between factbase and source code not visible again :-(
 +
  
teaching/labs/xp/2009a/test_framework.txt · Last modified: 2018/05/09 01:59 (external edit)

SEWiki, © 2019