iValidator is a framework for XML-based test automation of complex test scenarios. iValidator is completely written in Java. The framework is available under an open source licence.
For those of you who've got into it you'll know that test driven development is great. It gives you the confidence to change code safe in the knowledge that if something breaks you'll know about it. Except for those bits you don't know how to test. Until now XML has been one of them. Oh sure you can use "<stuff></stuff>".equals("<stuff></stuff>"); but is that really gonna work when some joker decides to output a <stuff/>? -- damned right it's not ;-)
Feedback is vital for the practice of Continuous Integration (CI) -- in fact, it's the life blood of a CI system. Rapid feedback enables speedy responses to build events that require attention. Without feedback mediums like e-mail or RSS, builds in a broken state have the tendency to stay broken, which defeats the purpose of CI in the first place! In this installment of Automation for the people, automation expert Paul Duvall examines various feedback mechanisms that you can incorporate into CI systems.
Phantastic!
"EclEmma is a free Java code coverage tool for Eclipse, available under the Eclipse Public License. Internally it is based on the great EMMA Java code coverage tool, trying to adopt EMMA's philosophy for the Eclipse workbench:
* Fast develop/test cycle: Launches from within the workbench like JUnit test runs can directly be analyzed for code coverage.
* Rich coverage analysis: Coverage results are immediately summarized and highlighted in the Java source code editors.
* Non-invasive: EclEmma does not require modifying your projects or performing any other setup.
The Eclipse integration has its focus on supporting the individual developer in an highly interactive way.
The update site for EclEmma is http://update.eclemma.org/."
Canoo WebTest is a free open source tool for automated testing of web applications.
It calls web pages and verifies results, giving comprehensive reports on success and failure. The White Paper provides an overview of the features and the design rationale. Detailed information is provided in the Manual Overview as well as the Install and Troubleshooting guides.
In what I hope will be the first of several articles about Guice, a new lightweight dependency injection container from Bob Lee and Kevin Bourillion from Google, this article examines the simplest and most obvious use case for the Guice container, for mocking or faking objects in unit tests. In future articles I will examine other, more ambitious areas where it can be used, including dependency elimination in large code bases.
The key to agility is being able to modify code easily and safely. The problem is that many Java applications are too brittle to extend and enhance easily. Attempts to fix or extend - no matter how carefully done - can introduce more bugs and more complexity.
With a full suite of characterization tests generated by JUnit Factory you can bring your legacy code under control. Download our free plug-in for Eclipse to get started.
AtUnit minimizes boilerplate code in unit tests and guides test development by enforcing good practices.
* mark exactly one field with @Unit to indicate the object under test.
* mark fields with @Mock or @Stub to obtain mock objects
* inject your tests, and your test subjects, using your favorite IoC container
Mock Objects Integration
AtUnit integrates with JMock or EasyMock to provide mock objects:
* obtain a JMock context simply by declaring a field
* annotate fields with @Mock to obtain JMock or EasyMock mock objects
* annotate fields with @Stub to obtain a JMock or EasyMock stub object
... or you can use your own mock objects plug-in with two easy steps:
* implement the MockFramework interface
* annotate your tests with @MockFrameworkClass(MyMockFramework.class)
Container Integration
AtUnit integrates with Guice or Spring to take all of the work out of dependency-injected tests.
With Guice:
* never see the Injector, never write bootstrapping boilerplate!
* @Inject test class fields without even defining a Module
* declaratively obtain mock objects with @Inject @Mock
* if you need more binding flexibility, simply have your test class implement Module
With Spring:
* annotate fields with @Bean to get them from the Spring context
* fields annotated with @Bean which do not appear in your Spring context are added to it automatically! (This includes @Mock and @Stub fields.)
* AtUnit looks for a Spring XML file with the same name as your test, or you can specify the location yourself with @Context("filename")
* Most of the time, you don't even need a Spring XML file!
You can easily plug in other containers in two steps:
* implement the Container interface
* annotate your tests with @ContainerClass(MyContainer.class)
Testability-explorer is a tool which analyzes java byte-codes and computes how difficult it will be to write unit-test. It attempts to help you quantitatively determine how hard your code is to test and, where to focus to make it more testable.
Test metric tool can be used:
1. As a learning tool which flags causes of hard to test code with detailed breakdown of reasons.
2. To identify hard to test hair-balls in legacy code.
3. As part of your code analysis-toolset.
4. As a tool which can be added into continuous integration that can enforce testable code.
This article demonstrates a brand new open source library for unit testing, called Unitils. Unitils helps you in writing simple and maintainable unit tests with JUnit or TestNG . It glues together some widely used test libraries like DbUnit and EasyMock and offers integration with Spring and Hibernate . Unitils encourages applying good practices and unit testing guidelines. The ideas behind the code are based on the authors' concrete experience on enterprise projects.
Unitils offers following features
* Equality assertion through reflection, with options like ignoring Java default/null values and ignoring order of collections
* Support for database testing involving test data management with DbUnit, automatic maintenance of unit test databases and automatic constraints disabling
* Hibernate integration features such as session management and testing the mapping with the database
* Integration with Spring, involving ApplicationContext management and injection of spring managed beans
* Integration with EasyMock and injection of mocks into other objects
RMock 2.0.0 is a Java mock object framework to use with jUnit. RMock has support for a setup-modify-run-verify workflow when writing jUnit tests. It integrates better with IDE refactoring support and allows designing classes and interfaces in a true test-first fashion.
Mock is a widely used technology in unit test domain. It shields exteral and unnecessary factors and helps developers focus on the specific function to be tested.
EasyMock is a well known mock tool which can create mock object for given interface at runtime. The mock object's behavior can be fine defined prior the test code in the test case. EasyMock is based on java.lang.reflect.Proxy, which can create dynamic proxy class/object according to the given interfaces. It has the inherent limitation from the Proxy. It can create mock object for given interface only.
Mocquer is a similar mock tool as EasyMock. With the help of Dunamis Project, it extends the function of EasyMock to support mock object creation for both class and interface.
With Marathon you capture user interactions on the applications and also insert assertions to verify that correct processing is taking place. The generated raw script can be refactored to modules for efficient reuse and maintainability. Replay the scripts either manually or integrate Marathon into your build process for automatic execution of the test suites.
Fitnesse for Eclipse Plugin
Add Fit and Fitnesse support to your Eclipse tooling environment!
The FitNesse for Eclipse Plugin enables developers to more easily use the FitNesse and Fit frameworks from within the Eclipse environment.
This is a very simple integration of Spring and FitNesse. FitnNesse is a Wiki-based framework to implement integration tests. Fit as well. This framework uses HTML files as input and output for the integration tests. Therefore the tests can be written using Word or Excel instead of a Wiki. This makes usage much easier and can be critical for success or failure.
This framework offers new classes that you can inherit from to implement or Fit FitNesse Fixtures. These Fixtures are then autowired to Spring Beans. So the Fixtures just need to implement set-methods. These are automatically called by Spring with a Spring Bean of the right type. This way Dependency Injection also covers the Fitnesse tests.
ClassMock is a framework that helps the creation of unit tests for components that use reflection or annotations. In this kind of classes, the behavior is dependent of the class structure. This way, each test case usually works with a different class created specifically for the test. With ClassMock is possible to define and generate classes in runtime, allowing a better test readability and logic sharing between tests.
This blog is written by the employees of Stelligent Incorporated. Stelligent experts help teams deliver better software faster by focusing on production over process. We are tools and methodology agnostic, so whether you are using .NET, Java, or a dynamic language environment, check out the Test Early blog. If you have questions, please contact admin@testearly.com
There are many Continuous Integration systems available. This page is an attempt to keep an unbiased comparison of as many as possible of them. The goals are:
* Make it easier to choose an appropriate CI tool for your project.
* "Healthy competition aid" for the people involved in the development of these various CI systems.
Nice article on IBM deveWorks: Ready to step up to the plate and hit a home run with your developer testing activities? In this installment of Automation for the people, development automation expert Paul Duvall covers some of the various types of automated developer tests you can run with every source code change. Paul provides examples of Selenium, DbUnit, and JUnitPerf tests that can help you discover application problems early -- that is, if they're run often.
TestLink is a open source web based TEST MANAGEMENT and test EXECUTION system under the GPL license (i.e. free to use). The tool enables quality assurance teams to create and manage their test cases as well as organize them into test plans. These test plans allow team members to execute test cases and track test results dynamically, generate reports, trace software requirements, prioritize and assign.
The tool is based on PHP, MySQL, and includes several other open source tools. We support interface to Bug tracking systems as is Bugzilla or Mantis.
Unitils is an open source library aimed at making unit testing easy and maintainable. Unitils builds further on existing libraries like DBUnit and EasyMock and integrates with JUnit and TestNG.
Unitils provides general asserion utilities, support for database testing, support for testing with mock objects and offers integration with Spring and Hibernate. It has been designed to offer these services to unit tests in a very configurable and loosely coupled way. As a result, services can be added and extended very easily.
Currently Unitils offers following features:
* General testing utilities
o Equality assertion through reflection, with different options like ignoring Java default/null values and ignoring order of collections
* Database testing utilities
o Automatic maintenance and constraints disabling of unit test databases
+ Support for Oracle, Hsqldb, MySql, DB2, Postgresql and Derby
o Simplify unit test database connection setup
o Simplify insertion of test data with DBUnit
o Simplify Hibernate session management for unit testing
o Automatically test the mapping of Hibernate mapped objects with the database
o Manage transactions during unit testing
* Mock object utilities
o Simplify EasyMock mock object creation
o Simplify mock object injection
o EasyMock argument matching using reflection equality
* Spring integration
o ApplicationContext configuration and easy injection of spring managed beans into a unit test
o Support for using a Spring-configured Hibernate SessionFactory in unit tests.
The project started begin 2006 from an Ordina J-Technologies discussion group on unit testing. The result was a list of guidelines and Unitils emerged in an attempt to write code to support these guidelines.
Documentation
ServiceFixture is a fixture library built on top of FIT/FitNesse which enables FitNesse to be the integration test and software collaboration platform for service oriented and distributed systems. These systems usually expose stateless services, such as web service (SOAP or REST), ejb and POJO etc, to its internal or external clients, and complex domain objects are always involved in the service interfaces.
It is difficult to leverage FitNesse for these systems due to the fact that it is not trivial to represent complex domain objects in FIT test table, and hence it would require constant fixture development efforts to support the integration test. ServiceFixture is designed to fill this gap.
ServiceFixture uses expression language to represent domain objects, verify and display the response in FIT test table. Every ServiceFixture test is basically a method invocation to a stateless service. The test starts with setting up the input data, then invoking the service operation, and then checking the response.
Because the ServiceFixture is very generic, to setup ServiceFixture and FitNesse for a project, you only need to write a ServiceFixture extension to encapsulate the invocation logic once, which is pretty trivial. No more fixture development is needed after that. It also provides template creator integrated with FitNesse to make tester's life a lot easier too.
ServiceFixture also provides database service fixtures(SelectFixture and UpdateFixture) which can be used to access databases directly from FitNesse test scripts. See Tutorial for more details.
FitLibrary provides general-purpose library fixtures (and runners) for Fit and FitNesse. It includes DoFixture, which provides an elegant way of organising storytests in general and of expressing workflow in particular.
With so many Continuous Integration (CI) servers to choose from, it can be difficult to decide which one is right for you. In the second article of the series Automation for the people, development automation expert Paul Duvall looks at a handful of open source CI servers, including Continuum, CruiseControl, and Luntbuild, using a consistent evaluation criteria and illustrative examples.
Dependometer is a java based analysis tool for java projects.
Features are:
Use a logical architecture description in terms of Layers and Subsystems and their physical mapping (n Packages implement a Subsystem) and check logical architecture violations.
* Analyze the dependency architecture between Layers, Subsystems, Packages, Compilation Units (Java files) and Types (Classes and Interfaces)
* Calculate a bunch of metrics for all elements - this includes metrics from John Lakos, Robert C. Martin and Craig Larman
* Analyze cycles between elements
* Define thresholds and receive feedback upon their violation
Simulate via simple refactoring definitions and cutting unwanted dependencies changes to the physical structure possibly enhancing refactoring.
Create a complete HTML presentation via xslt. This provides browsing capabilities from Layer (logical element) to Compilation Unit (physical element) for a discussion which physical elements cause the logical architecture to break.
Spring AutoMock is a test enabling framework to allow automatic exposure of Mocked beans for a Spring application. Used in conjunction with Spring autowiring of bean dependencies you can develop teired application contexts that represent the architectural tiers of your application, and thus you testing strategy. The simplest example is a separation of service beans and DAO beans into separate xml application contexts so that the services can be fully tested in isolation of the DAOs. The DAO beans are still required by the services typically as an injected property. Spring AutoMock can automatically register a Mock and a proxy of certain beans, so that the Mocks can be injected into your test cases and the matching proxy into the item under test. This reduces the need for repeated Spring test configuration.
The Grinder is a JavaTM load testing framework that makes it easy to run a distributed test using many load injector machines. It is freely available under a BSD-style open-source license.
The latest news, downloads, and mailing list archives can be found on SourceForge.net.
Key features
* Generic Approach Load test anything that has a Java API. This includes common cases such as HTTP web servers, SOAP and REST web services, and application servers (CORBA, RMI, JMS, EJBs), as well as custom protocols.
* Flexible Scripting Tests are written in the powerful Jython scripting language.
* Distributed Framework A graphical console allows multiple load injectors to be monitored and controlled, and provides centralised script editing and distribution.
* Mature HTTP Support Automatic management of client connections and cookies. SSL. Proxy aware. Connection throttling. Sophisticated record and replay of the interaction between a browser and a web site.
See the longer features list for further details.
This library allows you to use JavaBeans-style property matching for arguments when using EasyMock. Property matching is based on commons-beanutils as documented in Standard JavaBeans.
SevenMock is a light-weight Java dynamic mock objects framework. It is unusual in that it places responsibility for verifying operation parameters directly on the unit test code. This enables the test designer to write very clear, precisely targeted tests and makes test failures easier to diagnose.
UTF-X is a unit testing framework for XSLT. UTF-X strongly supports the test-first-design principle with test rendition and test validation features allowing you to visually design your test before you start working on the stylesheet. UTF-X was originally built to test XSLT stylesheets used in an XML publishing system so it has good support for DTD validation, XHTML and XSL:FO stylesheets.
JMock is a library that supports test-driven development1 of Java2 code with mock objects3.
Mock objects help you design and test the interactions between the objects in your programs.
The jMock library:
* makes it quick and easy to define mock objects, so you don't break the rhythm of programming.
* lets you precisely specify the interactions between your objects, reducing the brittleness of your tests.
* works well with the autocompletion and refactoring features of your IDE
* plugs into your favourite test framework
* is easy to extend.
This is the homepage of the Code Analysis Plugin (CAP).
CAP is a plugin for the eclipse platform and analysis the dependencies of your Java project. It opens a own perspective and displays the results in an clear way using different diagrams.
What if you were able to discover potential problems in your code prior to building it? Interestingly enough, there are Eclipse plugins for tools such as JDepend and CheckStyle that can help you discover problems before they are manifested in software. In this installment of Automation for the people, automation expert Paul Duvall provides examples of installing, configuring, and using these static analysis plugins in Eclipse so that you can prevent problems early in the development life cycle.
When starting new projects, most of us plan to review code before actually releasing it into production; however, when delivery schedules supersede other factors, reviews tend to be the first practice thrown out. What if you were able to perform a portion of these reviews automatically? In this first article of the new series Automation for the people, development automation expert Paul Duvall begins with a look at how automated inspectors like CheckStyle, JavaNCSS, and CPD enhance the development process and when you should use them.
JUnitPerf is a collection of JUnit test decorators used to measure the performance and scalability of functionality contained within existing JUnit tests.
There are a number of open-source unit testing tools available. So why another one?
Well, this one addresses a specific need - an easy way to test XML-based servers. If you have a server that communicates with clients via XML messages, you can end up putting a lot of effort into using one of the unit-testing frameworks to test all the messages. Just think of all the code needed to set up communications, construct messages, and verify responses.
A simple alternative is to document XML messages and expected responses, without having to write any code. Let XmlMessageTest send each XML message to the server, verify returned messages against expected results, and produce a simple report of test results.
XmlMessageTest is written in Java and should be able to run any Java-enabled platform. It's been tested on Windows XP and Linux. It can be easily integrated into your build process.
The developer edition provides ALL of the capabilities of the server edition but limits console and terminal connectivity to the first 45 minutes of a managed JVM's processing. Snapshots taken before the expiration time can still be analyzed offline in the console and the console will reconnect to the JVM once it has been stopped and started.
ScalaCheck is a powerful tool for automatic unit testing of Scala and Java programs. It features automatic test case generation and minimization of failing test cases. ScalaCheck started out as a Scala port of the Haskell library QuickCheck, and has since evolved and been extended with features not found in Haskell QuickCheck.
Abbot helps you test your Java UI. It comprises Abbot, which lets you programmatically drive UI components, and Costello (built on Abbot) which allows you to easily launch, explore and control an application. The framework may be used with both scripts and compiled code.
jDiffChaser is a GUI comparison tool that automates difference detection
between same screens of different versions. You can easily record scenarios
(optionally define zones of the screens to ignore during comparisons) and play suites
of them on two different versions of the same Java Swing application: differences are
then listed in a web page report.
p-unit
An open source framework for unit test and performance benchmark, which was initiated by Andrew Zhang, under Apache License v2.0. p-unit supports to run the same tests with single thread or multi-threads, tracks memory and time consumption, and generates the result in the form of plain text, image or pdf file.
Two powerful tools for communicating requirements from your customers, and testing those requirements, are Fitnesse and Selenium. Fitnesse, a wiki encapsulating the Framework for Integration Tests (aka FIT), enables customers to write sentence-like tests which can be mapped to the underlying system. Selenium drives a browser without all of the fragile mouse coordinate testing you get from a lot of testing tools (ala WinRunner).
How much time do you spend maintaining project build scripts? Probably much more than you'd expect or would like to admit. It doesn't have to be such a painful experience. Development automation expert Paul Duvall uses this installment of Automation for the people to demonstrate how to improve a number of common build practices that prevent teams from creating consistent, repeatable, and maintainable builds.
Luntbuild is a powerful build automation and management tool. Continuous Integration or nightly builds can be easily set using a clean web interface. Executed builds are well managed using functions such as search, categorization, promotion, patching, deletion, etc. It also acts as a central build artifacts repository and download area for your whole team.
JRat is the Java Runtime Analysis Toolkit. Its purpose is to enable developers to better understand the runtime behavior of their Java programs. The term "behavior" includes, but is not limited to performance profiling.
While JRat is still in beta, without adding code to your application it can...
# accumulate timing statistics (a few ways)
# create trace logging
# track rate methods are called over time
# track the response time of methods over time