Testing
What is Software Testing?
- Software testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results
- All the possible values need to be tested and verified, but complete testing is infeasible
- Software testing is a trade-off between budget, time and quality.
There are two basic classes of testing: Black box testing & White box testing. Black-box and white-box are test design methods.
- Black-box test design treats the system as a “black-box”, so it does not explicitly use knowledge of the internal structure. Black-box test design is usually described as focusing on testing functional requirements. Synonyms for black-box include: opaque-box, and closed-box.
- White-box test design allows one to peek inside the “box” and it focuses specifically on using internal knowledge of the software to guide the selection of test data. Synonyms for white-box include: structural, glass-box and clear-box.
Why Testing?
- Software Testing is the process of executing a program or system with the intent of finding errors
Quality assurance(To improve quality)
- The imperfection of human nature makes it almost impossible to make a moderately complex program correct the first time
- As software is used in critical applications, the outcome of a bug can be severe
- Quality means the conformance to the specified design requirement
For Verification and Validation
- Testing can serve as metrics in the V&V process
- Based on testing results, it is possible to predict that whether the product works under certain situations, or it does not work
- Tests with the purpose of validating the product works are named clean tests, or positive tests
- Tests aiming at breaking the software or showing that it does not work are named as dirty test or negative tests
Software Testing Strategies
A strategy for software testing integrates software test case design techniques into a well- planned set of steps that cause the production of software. A software test strategy provides a road map for the software developer, the quality assurance organisation, and the customer.
Any testing strategy needs to include test planning, test case design, test execution, and the resultant data collection evaluation
Unit Testing
- Unit testing concentrates verification on the smallest unit of the program – the module. In an object-oriented environment, this is usually at the class level, and the minimal unit tests include the constructors and destructors. Using the detailed design description important control paths are tested to establish errors within the bounds of the module. Unit Testing is a white box testing.
- Unit testing is typically seen as an adjunct to the coding step. Once source code has been produced, reviewed, and verified for correct syntax, unit test case design can start. A review of design information offers assistance for determining test cases that should uncover errors. Each test case should be linked with a set of anticipated results. As a module is not a stand-alone program, driver and/stub software must be produced for each test units. In most situations a driver is a “main program” that receives test case data, passes this to the module being tested and prints the results. Stubs act as the sub-modules called by the test modules. Unit testing is made easy if a module has cohesion.
- In extreme programming and the agile software development movement, adhere to a “test-driven software development” model. In this process, unit tests are written first, by the software engineers (often with pair programming in the extreme programming methodology). Of course these tests fail initially; as they are expected to. Then as code is written it passes incrementally larger portions of the test suites. The test suites are continuously updated as new failure conditions and corner cases are discovered, and they are integrated with any regression tests that are developed. Unit tests are maintained along with the rest of the software source code and generally integrated into the build process (with inherently interactive tests being relegated to a partially manual build acceptance process).
Scope of Unit Testing
- Module interface is tested to ensure that information properly flows into and out of the program unit being tested.
- The local data structure is considered to ensure that data stored temporarily maintains its integrity for all stages in an algorithm’s execution.
- Boundary conditions are tested to ensure that the modules perform correctly at boundaries created to limit or restrict processing.
- All independent paths through the control structure are exercised to ensure that all statements in been executed once.
- Finally, all error-handling paths are examined.
Functional Testing
- Functional testing covers how well the system executes the functions it is supposed to execute—including user commands, data manipulation, searches and business processes, user screens, and integrations. Functional testing covers the obvious surface type of functions, as well as the back-end operations (such as security and how upgrades affect the system). It is a black-box testing.
There are many types of functional testing:
- functionality testing
- input domain testing.
- error handling.
- database testing.
- object property checking
- compatibility testing
- configuration testing
- sanitation testing
- installation testing
- inter system testing.
Integration Testing
- Once all the individual units have been tested there is a need to test how they were put together to ensure no data is lost across interface.It can be black box testing as well as white box testing.
- Integration testing exposes defects in the interfaces and interaction between integrated components (modules). Progressively larger groups of tested software components corresponding to elements of the architectural design are integrated and tested until the software works as a system
- Integration defects can arise, when the new modules are developed in separate branches, and then integrated into the main project.
Top Down Integration & Its scope
- Top-down integration is an incremental approach to the production of program structure. Modules are integrated by moving downwards through the control hierarchy, starting with the main control module. Modules subordinate to the main control module are included into the structure in either a depth-first or breadth-first manner.
The integration process is performed in a series of five stages:
- The main control module is used as a test driver and stubs are substituted for all modules directly subordinate to the main control module.
- Depending on the integration technique chosen, subordinate stubs are replaced one at a time with actual modules.
- Tests are conducted as each module is integrated.
- On the completion of each group of tests, another stub is replaced with the real module.
- Regression testing may be performed to ensure that new errors have been introduced.
- The major disadvantage of top-down approaches is the need for stubs and the difficulties that are linked with them
Bottom Up Integration & Its scope
- Bottom-up integration testing, begins testing with the modules at the lowest level (atomic modules).As modules are integrated bottom up, processing required for modules subordinates to a given level is always available and the need for stubs is eliminated.
A bottom-up integration strategy may be implemented with the following steps:
- Low-level modules are combined into clusters that perform a particular software sub function.
- A driver is written to coordinate test cases input and output.
- The cluster is tested.
- Drivers are removed and clusters are combined moving upward in the program structure.
- The major drawback of bottom-up integration is that the program does not exist until the last module is included.
Acceptance Testing
A formal test conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept the system.Hence called User Acceptance testing.UAT is Black box testing.
What is User Acceptance Testing?
- User Acceptance Testing is often the final step before rolling out the application.
- Usually the end users who will be using the applications test the application before ‘accepting’ the application.
This type of testing gives the end users the confidence that the application being delivered to them meets their requirements.
This testing also helps nail bugs related to usability of the application.
User Acceptance Testing – Prerequisites:
- Before the User Acceptance testing can be done the application is fully developed.
- Various levels of testing (Unit, Integration and System) are already completed before User Acceptance Testing is done. As various levels of testing have been completed most of the technical bugs have already been fixed before UAT.
User Acceptance Testing – What to Test?
To ensure an effective User Acceptance Testing Test cases are created. These Test cases can be created using various use cases identified during the Requirements definition stage. The Test cases ensure proper coverage of all the scenarios during testing.
During this type of testing the specific focus is the exact real world usage of the application. The Testing is done in an environment that simulates the production environment. The Test cases are written using real world scenarios for the application
User Acceptance Testing – How to Test?
The user acceptance testing is usually a black box type of testing. In other words, the focus is on the functionality and the usability of the application rather than the technical aspects. It is generally assumed that the application would have already undergone Unit, Integration and System Level Testing.
However, it is useful if the User acceptance Testing is carried out in an environment that closely resembles the real world or production environment.
The steps taken for User Acceptance Testing typically involve one or more of the following:
- User Acceptance Test (UAT) Planning:
- As always the Planning Process is the most important of all the steps. This affects the effectiveness of the Testing Process. The Planning process outlines the User Acceptance Testing Strategy. It also describes the key focus areas, entry and exit criteria.
- Designing UA Test Cases:
- The User Acceptance Test Cases help the Test Execution Team to test the application thoroughly. This also helps ensure that the UA Testing provides sufficient coverage of all the scenarios.
- The Use Cases created during the Requirements definition phase may be used as inputs for creating Test Cases. The inputs from Business Analysts and Subject Matter Experts are also used for creating.
- Selecting a Team that would execute the (UAT) Test Cases:
- Selecting a Team that would execute the UAT Test Cases is an important step.
- The UAT Team is generally a good representation of the real world end users.
- The Team thus comprises of the actual end users who will be using the application.
- Executing Test Cases:
- The Testing Team executes the Test Cases and may additional perform random tests relevant to them.
- Documenting the Defects found during UAT:
- The Team logs their comments and any defects or issues found during testing.
- Resolving the issues/Bug Fixing:
- The issues/defects found during Testing are discussed with the Project Team, Subject Matter Experts and Business Analysts. The issues are resolved as per the mutual consensus and to the satisfaction of the end users.
- Sign Off:
- Upon successful completion of the User Acceptance Testing and resolution of the issues the team generally indicates the acceptance of the application. This step is important in commercial software sales.
- Once the User “Accepts” the Software delivered they indicate that the software meets their requirements.
Each User Acceptance Test Case describes in a simple language the precise steps to be taken to test something.
The Business Analysts and the Project Team review the User Acceptance Test Cases.
The users now confident of the software solution delivered and the vendor can be paid for the same.
Code Coverage
Code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that inspects the code directly and is therefore a form of white box testing. Code coverage analysis is the process of:
- Finding areas of a program not exercised by a set of test cases
- Creating additional test cases to increase coverage, and
- Determining a quantitative measure of code coverage, which is an indirect measure of quality.
- Identifying redundant test cases that do not increase coverage.
Use coverage analysis to assure quality of set of tests, not the quality of the actual product. You do not generally use a coverage analyzer when running your set of tests through your release candidate. Coverage analysis requires access to test program source code and often requires recompiling it with a special command. Coverage analysis is one of many testing techniques; you should not rely on it alone. Code coverage analysis is sometimes called test coverage analysis. The two terms are synonymous. The academic world more often uses the term “test coverage” while practitioners more often use “code coverage”. Likewise, a coverage analyzer is sometimes called a coverage monitor. I prefer the practitioner terms.
Coverage Criteria
To measure how well the program is exercised by a test suite, one or more coverage criteria are used. There are a number of coverage criteria, the main ones are listed here.
Statement coverage
Has each line of the source code been executed by our test? Statement coverage identifies which statements in a method or class have been executed. It is a simple metric to calculate, and several open source products exist that measure this level of coverage. Ultimately, the benefit of statement coverage is its ability to identify which blocks of code have not been executed. The problem with statement coverage, however, is that it does not identify bugs that arise from the control flow constructs in your source code, such as compound conditions or consecutive switch labels. That means you easily can get 100% coverage and still have glaring, uncaught bugs.
Loop coverage
Loop coverage metric reports whether you executed each loop body zero times, exactly once, and more than once (consecutively). For do-while loops, loop coverage reports whether you executed the body exactly once, and more than once. The valuable aspect of this metric is determining whether while-loops and for-loops execute more than once because this information not reported by other metrics.
Branch coverage
A branch is the outcome of a decision, so branch coverage simply measures which decision outcomes have been tested. This sounds great because it takes a more in-depth view of the source code than simple statement coverage, but branch coverage can also leave you wanting more. Determining the number of branches in a method is easy. Boolean decisions obviously have two outcomes, true and false, whereas switches have one outcome for each case and don't forget the default case! The total number of decision outcomes in a method is therefore equal to the number of branches that need to be covered plus the entry branch in the method (after all, even methods with straight line code have one branch).
Path coverage
A path represents the flow of execution from the start of a method to its exit. A method with N decisions has 2^N possible paths, and if the method contains a loop, it may have an infinite number of paths. Like branch coverage, testing the basis set of paths ensures that you test every decision outcome, but, unlike branch coverage, basis path coverage ensures that you test all decision outcomes independently of one another. In other words, each new basis path “flips” exactly one previously executed decision, leaving all other executed branches unchanged. This is the crucial factor that makes basis path coverage more robust than branch coverage, and allows you to see how changing that one decision affects the method's behaviour.
Testing with Ruby on Rails
One of the real joys of the Rails framework is that it has support for testing build in from the start of every project. Indeed, from the moment you create a new application using the rails command, Rails starts generating a test infrastructure for you.
Create test application
First we build a project by
$rails -D <database_type> <name_of_project> where you should specify your database type and the name of the project. By default Rails uses SQLite3, but in this application we will use PostgreSQL.
Create databases to be used by our new application:
$ sudo su postgres postgres$ createuser <your_username> Shall the new role be a superuser? (y/n) y exit $ psql template1 template1=# CREATE DATABASE rails_development > CREATE DATABASE template1=# CREATE DATABASE rails_test > CREATE DATABASE template1=# CREATE DATABASE rails_production > CREATE DATABASE template1=# \l # to list the databases
and configure config/database.yml appropriately
development: adapter: postgresql encoding: utf8 database: rails_development username: <your_username> pool: 5 test: adapter: postgresql encoding: utf8 database: rails_test username: <your_username> pool: 5 test: adapter: postgresql encoding: utf8 database: rails_production username: <your_username> pool: 5
Now let`s generate a model for us to test:
$ script/generate scaffold Product title:string description:text image_url:string price:decimal
$ rake db:migrate
Begin testing
If you look in the top-level directory of that project you’ll notice a subdirectory called test. Inside this directory you’ll see four existing directories and a helper file.
$ ls test > fixtures functional integration test_helper.rb unit
So our first decision, where to put tests, has already been made for us. The rails command simply creates the test directory structure. Then, every time you run the generate script to create a model or a controller, Rails creates a test file to hold a corresponding test stub.
By convention, Rails calls things that test models unit tests and things that test controllers functional tests. Let’s take a peek inside the unit and functional subdirectories to see what’s already there.
$ ls test/unit > product_test.rb $ ls test/functional > products_controller_test.rb
Rails has already created files to hold the unit tests for the models and the functional tests for the controllers previously created with the generate script. This is a good start, but Rails can help us only so much. It puts us on the right path, letting us focus on writing good tests. We’ll start back where the data lives and then move up closer to where the user lives.
Testing Models
Testing database applications can be a serious pain. It’s made worse when database access code is sprinkled throughout the application. You can never seem to test the smallest chunk of code without first firing up the database and then spoon-feeding it enough data to make the code do something interesting. We programmers have a marketing term for that — bad coupling.
Rails promotes good testing (and good design) by enforcing a structure for your application whereby you create models, views, and controllers as separate chunks of functionality. All the application’s state and business rules that apply to its state are encapsulated in models. And Rails makes it easy to test models in isolation, so let’s get right to it.
First test
Let’s see what kind of test goodies Rails generated inside the file test/unit/product_test.rb when we created that model.
require File.dirname(__FILE__) + '/../test_helper' class ProductTest < ActiveSupport::TestCase # Replace this with your real tests. def test_truth assert true end end
We see here a test method called test_truth(), with a comment above it hint-
ing that we have work to do. But before we break a sweat, let’s just see if
the test passes.
$ ruby test/unit/product_test.rb Loaded suite test/unit/product_test Started E Finished in 0.091436 seconds. 1) Error: test_truth(ProductTest): ActiveRecord::StatementInvalid: PGError: ERROR: relation "products" does not exist : DELETE FROM "products" . . . 1 tests, 0 assertions, 0 failures, 1 errors
The test failed, but thankfully, it leaves us a clue — it couldn’t find the products database table. But we know there is one because we created one and we can manually test the application using a web browser.
A database just for tests
Remember that we created three databases for our application? It seemed like overkill at the time. One was for development use — the one you’ll be using when building the application. One was for production use — we hope that happens someday soon. And one was for testing. Rails unit tests automatically use the test database, and there are no products in it yet. In fact, there are no tables in it yet.
So, as a first step, let’s load our schema into the test database. It turns out there are two ways of doing this, assuming the test database has already been created. One one hand you could instruct rake to migrate the database to the test environment by executing:
$ rake environment RAILS_ENV=test db:migrate
If, however, you’ve been building the schema in the development database by hand, then you might not have a valid create script. Rails has a handy mechanism for cloning the structure (without the data) from the development database into the test database. Simply issue the following command in your application’s top-level directory:
$ rake db:test:clone_structure
Now have a schema in our test database, but we still don’t have any products. We could enter data by hand, perhaps by typing some SQL insert commands, but that’s tedious and somewhat error prone. And if we later write tests that modify the data in the database, we’ll somehow have to get our initial data reloaded before we can run tests again. Rails has the answer — test fixtures.
Test Fixtures
The word fixture means different things to different people. In the world of Rails, a test fixture is simply a specification of the initial contents of a model. So, if we want to ensure that our products table starts off with the correct contents at the beginning of every unit test, we need only specify those contents in a fixture and Rails will take care of the rest.
You specify the fixture data in files in the test/fixtures directory. These files contain test data in either Comma-Separated Value (CSV) format or YAML format. For our tests we’ll use YAML, as it’s preferred. Each YAML fixture file contains the data for a single model. The name of the fixture file is significant; the base name of the file must match the name of a database table. As we need some data for a Product model, which is stored in the products table, we create a file called products.yml. (Rails created an empty version of this fixture file when it generated the corresponding unit test.)
The format of the fixture file is straightforward:
book: id: 1 title: Agile Development description: Develop with Ruby on Rails image_url: http://.../book.jpg price: 19.99 hp_multi: id: 2 title: HP Office description: Multifunctional office printer/scanner from HP image_url: http://.../multi.jpg price: 29.99
Now that we have a fixture file, we want Rails to load up the test data into the products table when we run the unit test:
fixtures :products
Create and Read
Before running the ProductTest test case, let’s create some meaningful tests. We should rename test_truth() to test_create() to better explain what we’re testing. Next, add a method to setup a @product and in test_create(), we check that the @product from setup() matches the corresponding fixture data.
require File.dirname(__FILE__) + '/../test_helper' class ProductTest < ActiveSupport::TestCase fixtures :products def setup @product = Product.find(1) end def test_create assert_kind_of Product, @product assert_equal 1, @product.id assert_equal "Agile Development", @product.title assert_equal "Develop with Ruby on Rails", @product.description assert_equal "http://.../book.jpg", @product.image_url assert_equal 19.99, @product.price end end
Now that we have everything hooked up, a test and data to run it against, we can try our test case:
$ ruby test/unit/product_test.rb Loaded suite test/unit/product_test Started . Finished in 0.066849 seconds. 1 tests, 6 assertions, 0 failures, 0 errors
This may not seem like much, but it actually tells us a lot: the test database is properly configured, the products table is populated with data from the fixture, Active Record was successful in fetching a given Product object from the test database, and we have a passing test that actually tests something.
Update
OK, so the previous test verified that the fixture created a Product that could be read from the database. Now let’s write a test that updates a Product.
def test_update assert_equal 19.99, @product.price @product.price = 88.88 assert @product.save, @product.errors.full_messages.join("; ") @product.reload assert_equal 88.88, @product.price end
Destroy
Destroying a Product model object deletes the corresponding row from the database. Attempting to find the Product causes Active Record to throw a RecordNotFound exception. We can test that, too.
def test_destroy @product.destroy assert_raise(ActiveRecord::RecordNotFound) { Product.find(@product.id) } end
Validation
Our Product model does not do any validation at the moment, but this is about to change. We will add the following, easy to understand, code to the Product model in app/models/product.rb:
class Product < ActiveRecord::Base validates_presence_of :title, :description, :price validates_uniqueness_of :title validates_numericality_of :price protected def validate errors.add(:price, "should be positive") unless price.nil? || price > 0.0 end end
The Product now model validates, among other things, that the price is greater than zero. Only then will Active Record save the product away in the database. But how are we going to test this bit of validation?
def test_validate assert_equal 19.99, @product.price @product.price = 0.00 assert !@product.save assert_equal 1, @product.errors.count assert_equal "should be positive", @product.errors.on(:price) end
If the price is less than or equal to zero, then the Product isn’t saved in the database and a message is added to the errors list.
Keeping Tests Flexible
Duplicating information in the tests that’s already specified in the fixture file makes for brittle tests. Thankfully, Rails makes it easy to keep test data in one place—the fixture.
When a fixture is loaded, it’s put into a Hash object referenced by an instance variable of the test case. For example, the :products fixture is conveniently loaded into the @products instance variable. That way, instead of hard-coding the expected values in our tests, we can access the test data using hash semantics.
def test_read_with_hash assert_kind_of Product, @product fix_product = products(:book) assert_equal fix_product.id, @product.id assert_equal fix_product.title, @product.title assert_equal fix_product.description, @product.description assert_equal fix_product.image_url, @product.image_url assert_equal fix_product.price, @product.price end
Testing Model Business Rules
Up to this point we’ve tested that Rails works as advertised. What’s more likely (and humbling) is that we’ll add custom business rules to models, and our application will “go off the rails”. Tests make us, mere mortals, look like programming superheros.
Let`s suppose we want to see only products currently in stock. As we din not account for this functionality in the beginning, we will have to add it to our model. First, the generate scaffold command which we first used created a schema migration file in db/migrate folder and named it ending with create_products.rb. Locate this file and add our new attribute(column) stock by adding the following line:
t.integer :stock
Now we have to update the database with our new schema. This will drop the table and recreate it so you will lose any records previously entered. We will also update the test database, this time by cloning the development one.
$ rake db:migrate:reset
$ rake db:test:clone_structure
The schema has changed, so our scaffold code is now out-of-date. As we’ve made no changes to the code, it’s safe to regenerate it. Notice that the generate script prompts us when it’s about to overwrite a file. We type
a to indicate that it can overwrite all files.
$ script/generate scaffold Product title:string description:text image_url:string price:decimal stock:integer -f
helper :all in app/cotrollers/application.rb. You may uncomment it afterwards.
As the final touch we shall add a method responsible for finding the items in stock to our Product model.
Add this code to app/models/product.rb
def self.stock_items find(:all, :conditions => "stock > 0", :order => "title desc") end
Now we can get back to our model testing and design some test cases. First, add the following fixture for a product not currently in stock and don`t forget to put the stock attribute to the other fixtures:
not_in_stock: id: 3 title: Product not in stock description: A product with stock 0 image_url: http://.../book.jpg price: 99.99 stock: 0
And here is the test case:
def test_stock_items items = Product.stock_items assert_equal 2, items.length assert items[0].stock > 0 assert items[1].stock > 0 assert !items.include?(@not_in_stock) end
Dynamic Fixtures
Sometimes you need the test data in a fixture to change based on a condition or a change in the test environment. Or you simply want to generate data instead of manually typing it.
Both the YAML and CSV fixture formats are pre-processed by ruby when you load fixtures. This allows you to use Ruby to help you generate some sample data.
<% for i in 1..100 %> product_<%= i %>: id: <%= i %> title: product_name_<%= i %> description: Description for product <%= i %> image_url: http://.../product_image<%= i %>.jpg price: <%= i+0.99 %> stock: <%= (i%2)*i %> <% end %>
Dynamic fixtures are more often used to create a larger number of almost identical objects. The above example creates 100 test fixtures in just a few lines of code. This is also convenient for performance testing.
Testing Controllers
Controllers direct the show. They receive incoming web requests (typically user input), interact with models to gather application state, and then respond by causing the appropriate view to display something back to the user. So when we’re testing controllers, we’re making sure that a given request is answered with an appropriate response. We still need models, but we already have them covered with unit tests.
Rails calls things that test controllers functional tests. If you have a look in the test/functional/products_controller_test.rb file you will see that rails already generated some testing code along the way.
Let's discuss what is already there. First, we have a test case method for every action:
def test_should_get_index get :index assert_response :success assert_not_nil assigns(:products) end
The above code is the first generated test case which tests the index page to be reachable and have the products object. The first line simulates a HTTP GET request and sets the response. The following lines verify that the response is a success and it has the products object.
We have learned some handy assertions, but let’s look at some of the Rails-specific conveniences for testing controllers.
HTTP Request Methods
The following methods simulate an incoming HTTP request method of the same name and set the response.
- get()
- post()
- put()
- delete()
- head()
Each of these methods takes the same four parameters. Let’s take a look at get(), as we already encountered it:
get(action, parameters = nil, session = nil, flash = nil)
Executes an HTTP GET request for the given action and sets the response. The parameters are as follows.
- action: the action of the controller being requested
- parameters: an optional hash of request parameters
- session: an optional hash of session variables
- flash: an optional hash of flash messages
Example:
get :show, :id => products(:book).id
Custom assertions
In addition to the standard assertions provided by Test::Unit, functional tests can also call custom assertions after executing a request. We’ll be using the following custom assertions:
assert_response(type, message=nil)
Asserts that the response is a numeric HTTP status or one of the following symbols. These symbols can cover a range of response codes (so :redirect means a status of 300–399).
- :success
- :redirect
- :missing
- :error
assert_redirected_to(options = {}, message=nil)
Asserts that the redirection options passed in match those of the redirect called in the last action. You can also pass a simple string, which is compared to the URL generated by the redirection.
assert_template(expected=nil, message=nil)
Asserts that the request was rendered with the specified template file.
assert_tag(conditions)
Asserts that there is a tag (node) in the body of the response that meets all of the given conditions. (I affectionately refer to this assertion as the chainsaw because of the way it mercilessly hack through a response.) The conditions parameter must be a hash of any of the following optional keys:
- :tag: a value used to match a node’s type
- :content: a value used to match a text node’s content
- :attributes: a hash of conditions used to match a node’s attributes
- :parent: a hash of conditions used to match a node’s parent
- :child: a hash of conditions used to match at least one of the node’s immediate children
- :ancestor: a hash of conditions used to match at least one of the node’s ancestors
- :descendant: a hash of conditions used to match at least one of the node’s descendants
- :children: a hash for counting the children of a node, using any of the following keys:
- :count: a number or a range equalling (or including) the number of children that match
- :less_than: the number of children must be less than this number
- :greater_than: the number of children must be greater than this number
- :only: a hash (yes, this is deep) containing the keys used to match when counting the children
Variables
After a request has been executed, functional tests can make assertions against the following variables:
- assigns(key=nil) Instance variables that were assigned in the last action. The assigns hash must be given strings as index references.
- session A hash of objects in the session.
- flash A hash of flash objects currently in the session.
- cookies A hash of cookies being sent to the user.
- redirect_to_url The full URL that the previous action redirected to.
Running the test cases we notice that there is one failed test case:
$ ruby test/functional/products_controller_test.rb Loaded suite test/functional/products_controller_test Started F...... Finished in 0.157259 seconds. 1) Failure: test_should_create_product(ProductsControllerTest)
As you can see the error is self explanatory and it tells us that a product could not be created. This was to be expected as our validation rules prohibits creation of empty products. Below is a proper test case for this scope:
def test_should_create_product assert_difference('Product.count') do post :create, :product => { :title => "TestProduct", :description => "Description of TestProduct", :image_url => "http://.../test_product.jpg", :price => "19.99", :stock => "0" } end
Testing with Cucumber
Prerequisites:
- Rails version 2.1.0 or higher
- Cucumber [required]
Installation
You can use both Rubygems or Rails’ plug-in mechanism. Usually Rubygems is recommend. We will use this case scenario:
$ [sudo] gem install rspec rspec-rails cucumber webrat
Install dependencies
The plugins’ dependencies should be installed automatically, but if they are not here is what you need:
$ [sudo] gem install term-ansicolor treetop diff-lcs nokogiri builder
nokogiri gem by executing in bash [sudo] apt-get install libxml2-dev libxslt1-dev
Cucumber on Rails
You can create the necessary files in your rails project by:
ruby script/generate cucumber --spork
The –spork option is recommended (but no mandatory) so you can run your features faster (you should have the spork gem installed also: [sudo] gem install spork).
This generates a new rake task called features and also the necessary files in the features directory.
Start a feature
This example assumes a new project (from the one used above) You can use the feature generator to generate the first few features:
$ script/generate feature Products title:string description:text
This will generate a simple plain text feature with associated steps. Don’t get addicted to this generator – you’re better off writing these by hand in the long run.
Running a feature
If you haven’t written any code yet (this should be the case here), we'll have to write some, or generate some. Try this:
$ script/generate rspec_scaffold Product title:string description:text
rake db:migrate
rake features
This should output something like:
Feature: Manage products In order to [goal] [stakeholder] wants [behaviour] Scenario: Register new product # features/manage_products.feature:6 Given I am on the new product page # features/step_definitions/webrat_steps.rb:6 When I fill in "Title" with "title 1" # features/step_definitions/webrat_steps.rb:26 And I fill in "Description" with "description 1" # features/step_definitions/webrat_steps.rb:26 And I press "Create" # features/step_definitions/webrat_steps.rb:14 Then I should see "title 1" # features/step_definitions/webrat_steps.rb:114 And I should see "description 1" # features/step_definitions/webrat_steps.rb:114 Scenario: Delete product # features/manage_products.feature:16 Given the following products: # features/step_definitions/product_steps.rb:1 | name | description | | name 1 | description 1 | | name 2 | description 2 | | name 3 | description 3 | | name 4 | description 4 | When I delete the 3rd product # features/step_definitions/product_steps.rb:5 Then I should see the following products: # features/step_definitions/product_steps.rb:12 | Name | Description | | name 1 | description 1 | | name 2 | description 2 | | name 4 | description 4 | 2 scenarios (2 passed) 11 steps (11 passed) 0m0.344s
We have now done a basic test with cucumber. For more detailed information and techniques please visit The Cucumber Wiki
See also
External links
- Ruby on Rails on Wikipedia
- More about Software testing on Wikipedia
- Testing Railcasts These screencasts are short and focused on Ruby on Rails techniques
- Testing with Cucumber, a high-level testing framework
- More on cucumber Guide to refractoring complex scenarios

