Added by Michael Baessler, last edited by Michael Baessler on Apr 04, 2008
()
Test Plan for UIMA Version 2.2.2
This page documents the planned testing for the 2.2.2 release.
Test Schedule
| Date(s) | Task |
|---|---|
| March. 7 |
Code Freeze, bug fixes only |
| March. 10 |
Start test phase |
| March. 21 |
End of test, start release vote |
| March. 24 |
Start vote on general@incubator.a.o |
| April. 7 |
Target release date |
Documentation Review
Check Jira issues for necessary documentation updates:
| Key | Summary | Assignee | Documentation Update? |
Status | Res |
| UIMA-680 | CAS is not unlocked on Errors |
Thilo Goetz | no update needed | closed | Fixed |
| UIMA-733 | it is possible to load a type system descriptor that redefines the super type of the DocumentAnnotation | Thilo Goetz |
no update needed | closed |
Fixed |
| UIMA-814 | PEAR verification should be able to treat customResoruceSpecifiers |
Michael Baessler | no update needed |
closed |
Fixed |
| UIMA-718 | add featurePath helper class |
Michael Baessler | No updates necessary - Javadoc |
closed |
Fixed |
| UIMA-721 | Improve performance of ResultSpecification, especially for Capability Language Flows |
Marshall Schor | updated | closed |
Fixed |
| UIMA-782 | Document Java 1.5 requirement for running Eclipse to use CDE, and mark runtime plugin (and others) as needing 1.5 level |
Marshall Schor | done | closed |
Fixed |
| UIMA-802 | CDE is unable to create PEAR descriptor as delegate |
Marshall Schor |
no update needed | closed |
Fixed |
| UIMA-282 | Work well with Apache logging (Log4J) |
Michael Baessler | done | closed |
Fixed |
| UIMA-749 | add performance report to CVD |
Thilo Goetz | done | closed |
Fixed |
UIMA framework tests
| Component |
Test case | Description |
Tester |
Platform |
Test level name | % complete | Comments |
|---|---|---|---|---|---|---|---|
| Installation | UIMA_uimaj_Install_001 |
Installation verification testInstall release package on a clean machine, check if the path adjustment works correctly |
Michael |
Linux Windows |
uimaj-2.2.2-01 uimaj-2.2.2-04 uimaj-2.2.2-05 |
100% 100% |
|
| Installation | UIMA_uimaj_Install_002 |
Try migration, following documentation and using tool, with existing components |
Michael | Linux Windows |
uimaj-2.2.2-03 | 100% 100% |
Tested with an annotator component |
| Installation | UIMA_uimaj_Install_003 |
Use the UIMA src distribution package, build the source code and run the test cases for all components. Use different JVMs on different plattforms. Test with different JVM vendors and JVM versions (Java 1.4, Java 5 and Java 6) |
Michael |
Linux Windows |
uimaj-2.2.2-01 uimaj-2.2.2-04 uimaj-2.2.2-05 |
100% 100% |
Tested with different JVMs v5 and v6 |
| CPM |
UIMA_uimaj_CPM_001 |
Running a CPE with AE in Integrated Deployment Mode |
Michael |
Linux Windows |
uimaj-2.2.2-03 | 100% 100% |
Tested cpeGUI with examples using different JVMs |
| CPM |
UIMA_uimaj_CPM_002 |
Running a CPE with AE in Local Deployment Mode |
Michael | Linux Windows |
uimaj-2.2.2-03 uimaj-2.2.2-04 |
100% 100% |
UIMA-904 (won't be fixed) |
| CPM | UIMA_uimaj_CPM_003 |
Running a CPE with AE in Remote Deployment Mode |
Michael |
Linux Windows |
uimaj-2.2.2-03 |
100% 100% |
Tested cpeGUI with cpe non-managed example |
| CPM | UIMA_uimaj_CPM_004 |
Runing a CPE with mixed deployment modes | Michael |
Linux Windows |
uimaj-2.2.2-03 uimaj-2.2.2-04 |
100% 100% |
UIMA-904 (won't be fixed) |
| CPM | UIMA_uimaj_CPM_005 |
Introduce a TAE that crashes. Test CPM error handling options |
|
Linux Windows |
|
|
|
| Core | UIMA_uimaj_Core_001 |
UIMA remoting with SOAP Test UIMA remote analysis engines using the SOAP protocol |
Michael |
Linux Windows |
uimaj-2.2.2-03 uimaj-2.2.2-04 |
100% 100% |
UIMA-906 (fixed) |
| Core | UIMA_uimaj_Core_002 |
UIMA remoting with Vinci Test UIMA remote analysis engines using the Vinci protocol 1) Test the provided Vinci examples from the UIMA SDK on the local machine. (start Vinci server, deploy Vinci service, access Vinci service) 2) Access a deployed Vinci service from a remote maschine. |
Michael |
Linux Windows |
uimaj-2.2.2-03 | 100% 100% |
|
| Core | UIMA_uimaj_Core_003 |
Run SofA primitive and aggregate analysis engines, run using CPE with CollectionReader, CasInitializer, CasConsumer. Test sofa mapping. Test sofa creation, defaulting, multi-sofa Cas operation. Run as simple aggregate, as well as CPE. |
|
Linux Windows |
|
100% |
all Sofa testing activities are already covered by automated tests. |
| Core | UIMA_uimaj_Core_004 |
Test CAS multiplier component |
Michael | Linux Windows |
uimaj-2.2.2-05 |
100% 100% |
Just tested provided UIMA example |
| Core | UIMA_uimaj_Core_005 |
Test UIMA custom flow using the FlowController |
Michael |
Linux Windows |
uimaj-2.2.2-05 | 100% 100% |
Just tested provided UIMA examples |
| Core | UIMA_uimaj_core_006 |
Test UIMA PEAR runtime 1) install PEARs with the PEAR installer and use the created PEAR descriptor file to run the installed PEAR package (use a PEAR package that needs CLASSPATH and DATAPATH settings; use a PEAR package that use JCAS cover classes) 2) create and aggregate analysis engine descriptor where some of the delegates are PEAR descriptor files from the installed PEAR packages (use PEAR packages where two PEAR files modifies the DocumentAnnotation in different ways using JCAS classes) |
Michael |
Windows Linux |
uimaj-2.2.2-01 |
100% 100% |
|
| Tooling | UIMA_uimaj_Tooling_001 |
Test Cas Visual Debugger | Thilo |
Windows, Linux |
uimaj-2.2.2-04 |
100% 100% |
Found issue with performance report in UIMA core. Will fix later. On Linux only tested Java 1.6. |
| Tooling | UIMA_uimaj_Tooling_002 |
Test Document Analyzer |
Michael |
Linux Windows |
uimaj-2.2.2-03 uimaj-2.2.2-04 |
100% 100% |
UIMA-898 (fixed) UIMA-899 (fixed) |
| Tooling | UIMA_uimaj_Tooling_003 |
Test Pear Installer |
Michael |
Linux Windows |
uimaj-2.2.2-01 uimaj-2.2.2-03 |
100% 100% |
|
| Tooling | UIMA_uimaj_Tooling_004 |
Test Pear Merger 1) Merge two or more PEAR files using the command line PEAR merger tooling. Install and run the merged PEAR file with the PEAR installer. Run the PEAR file using the created PEAR descriptor. 2) Merge two PEAR files where both use JCAS cover classes. 3) Merge two PEAR files where both have different UIMA DATAPATH settings. 4) Merge two PEAR files where one of them has a $main_root variable in the descriptor settings. |
Michael |
Windows Linux |
uimaj-2.2.2-03 | 100% 100% |
|
| Tooling | UIMA_uimaj_Tooling_005 |
Test Annotation Viewer |
Michael |
Windows Linux |
uimaj-2.2.2-03 uimaj-2.2.2-04 |
100% 100% |
|
| Tooling | UIMA_uimaj_Tooling_006 |
Test CPE Configurator |
Michael |
Linux Windows |
uimaj-2.2.2-04 |
100% 100% |
I think this is cpeGUI, this was tested when doing the CPM tests above. |
| Tooling | UIMA_uimaj_Tooling_007 |
Test command line pear packager Test all three command line packaging modes. Use for the packaging test a component that has classpath, datapath and env var settings. 1) create a complete PEAR package 2) just create the PEAR installation descriptor 3) just package a PEAR file |
Michael |
Linux Windows |
uimaj-2.2.2-01 | 100% 100% |
|
| EclipsePlugins | UIMA_uimaj_EclipsePlugins_001 |
Test CDE (Component Descriptor Editor) |
Marshall | Eclipse 3.3 Linux Windows |
rc05 | 100% 100% |
|
| EclipsePlugins | UIMA_uimaj_EclipsePlugins_002 | Test JCasGen generation | Marshall | Eclipse 3.3 Linux Windows |
rc05 | 100% 100% |
|
| EclipsePlugins | UIMA_uimaj_EclipsePlugins_003 | Test Pear packager 1) Test pear packaging with classpath, datapath and environment variable settings. |
Michael |
Eclipse 3.2 Eclipse 3.3 Windows Linux |
uimaj-2.2.2-04 uimaj-2.2.2-05 |
100% 100% |
tested on Windows with eclipse 3.2, 3.3 tested on Linux with eclipse 3.2 and 3.3. |
| EclipsePlugins | UIMA_uimaj_EclipsePlugins_004 |
Test UIMA Update Site for Eclipse. Install into 3.2 and 3.3 versions using update mechanism. Verify dependencies are loaded (install before EMF, verify EMF is loaded). Try both large and small Eclipse downloads. |
Marshall | Eclipse 3.2.1 Eclipse 3.3.0 Windows Linux |
|||
| Examples |
UIMA_uimaj_Examples_001 |
Test provided UIMA examples |
Jerry Cwiklik | Linux Windows |
uimaj-2.2.2-03 uimaj-2.2.2-04 uimaj-2.2.2-05 |
80% 100% |
Tested vinci, soap, most AEs, casMultiplier, FlowControler, Tutorials, CPE, some CasConsumers and CollectionReaders when doing the tests above. |
| |
|
|
|
|
|
|
Documentation
No special documentation testing/reading for this bugfix release necessary. The important documentation changes are covered by the
JIRA issue documentation review list above.