CDC Testing
This page summarizes the status of Derby testing on J2ME/CDC/Foundation Profile using JSR169 (referred as CDC in rest of the document).
Goal
The goal is to have all tests that can be run in CDC to be enabled. There will be a subset of tests which cannot be run in CDC because of platform/JSR169 limitations. A part of the goal is to identify and list such tests with valid reason for their exclusion.
Plan
To achieve this goal:
- Exclude tests which fail in CDC. This is to ensure a clean run of derbyall in CDC. Specify the reason for exclude, if known. Otherwise, indicate that this needs further investigation.
- For tests that can be run (at least partially) in CDC, modify the test/master files or split the test into two parts - for CDC and non-CDC environments. Some test failures could be possible bugs. Open JIRA issues for those.
- For tests that cannot be run, list a valid reason.
Status
Current CDC Test Coverage
The numbers are based on tests run with IBM's WCTME5.7 j9 foundation jvm. Numbers may be approximate.
Table of tests run in CDC by suite
- Total - Number of tests run/Total number of tests in the suite
- Skipped - Number of tests which are skipped using excludes
- To be enabled - Number of tests which have to be enabled to run in CDC
- Cannot be run - Number of tests which need to be skipped because of platform/JSR169 limitations.
Test Suite |
Total |
Skipped |
To be enabled |
Cannot be run |
Comments |
derbylang |
135/150 |
15 |
0 |
15 |
Listed in table below |
propertyinfo |
0/1 |
1 |
0 |
1 |
lang/GetPropertyInfoTest.java test uses Driver.getPropertyInfo method |
storeall |
90/97 |
7 |
0 |
7 |
Listed in table below |
storeunit |
33/34 |
1 |
0 |
1 |
unit/T_RawStoreFactory.unit |
unit |
3/4 |
1 |
1 |
|
unit/cacheService.unit hangs |
jdbcapi |
19/23 |
4 |
1 |
3 |
jdbcapi/checkDataSource30.java could be enabled |
jdbc20 |
9/10 |
1 |
0 |
1 |
jdbcapi/getCurConnJdbc20.sql |
jdk14 |
6/10 |
4 |
0 |
4 |
Listed in table below |
nist |
126/126 |
0 |
0 |
|
|
derbytools |
7/14 |
7 |
3 |
4 |
3 tests can be possibly re-written |
i18nTest |
5/7 |
2 |
0 |
2 |
Listed in table below |
Total |
90.9% (433/476) |
9.1% |
1.1% |
8% |
|
Goal: To be enabled=0%. Do one of the following for each test which is skipped.
- Enable the tests to run in CDC.
- Add to list that cannot be run specifying reason.
Test suites which cannot be currently run in CDC
The following table lists the suite that cannot be currently run in CDC.
Test Suite |
Comments |
derbynetclientmats |
Network Server not supported in CDC |
derbynetmats |
Network Server not supported in CDC |
derbynetautostart |
Network Server not supported in CDC |
xa |
XA not supported in CDC |
encryptionAll |
encryption not supported in CDC |
encryption |
encryption not supported in CDC |
jdbc40 |
JSR169 is based on JDBC 3.0 |
encodingTests |
Tests run only with jdk1.5 |
upgrade |
Tests require jdk1.4 or higher |
demo |
demo uses Driver/DriverManager classes |
simpledemo |
uses Driver/DriverManager classes |
Tests which cannot be currently run in CDC
The following table lists tests which cannot be run in CDC. These must be tests that exclusively test features that are not supported by CDC/JSR169.
Test Suite |
Test Name |
Reason |
jdbc20 |
jdbcapi/getCurConnJdbc20.sql |
Tests jdbc20 statement using server-side JDBC. JSR169 is based on JDBC3.0 |
jdbcapi |
jdbcapi/bestrowidentifier.sql |
Tests call to getBestRowIdentifier method using server-side JDBC |
jdbcapi |
jdbcapi/secureUsers1.sql |
Test uses ldap which is not supported in j9_foundation |
jdbcapi |
jdbcapi/checkDriver.java |
Test java.sql.Driver class |
jdk14 |
jdbcapi/checkDataSource30.java |
Can try to include this test |
jdk14 |
jdbcapi/parameterMetaDataJdbc30.java |
Tests ParameterMetaData - not available in JSR169 |
jdk14 |
lang/procedureJdbc30.java |
Tests procedures with server-side JDBC |
jdk14 |
jdbcapi/savepointJdbc30_XA.java |
savepointJdbc30 was split into two. Tests which can be run with JSR169 are in savepointJdbc30_JSR169 |
derbylang |
lang/authorize.sql |
test uses server-side JDBC - exclude for now |
derbylang |
lang/connect.sql |
test uses ij 'driver' command - exclude for now |
derbylang |
lang/errorCode.sql |
test uses ij 'driver' command - exclude for now |
derbylang |
lang/GetPropertyInfoTest.java |
test uses Driver.getPropertyInfo method |
derbylang |
lang/nestedCommit.sql |
test uses procedure with server-side JDBC |
derbylang |
lang/unaryArithmeticDynamicParameter.java |
uses jdk14 jdbc metadata calls |
derbylang |
lang/dcl.sql |
uses server-side JDBC - exclude for now |
derbylang |
lang/declareGlobalTempTableJavaJDBC30 |
test requires javax.naming.Referenceable |
derbylang |
lang/functions.sql |
test uses server-side JDBC - exclude for now - can be written as Java test |
derbylang |
lang/holdCursorIJ.sql |
Holdability can not be changed using jdbc3.0 apis in JDKs lower than JDK1.4. |
derbylang |
lang/releaseCompileLocks.sql |
uses server-side JDBC |
derbylang |
lang/procedure.java |
uses server-side JDBC |
derbylang |
lang/cast.sql |
test requires java.math.BigDecimal |
derbylang |
lang/triggerRefClause.sql |
test requires java.math.BigDecimal |
derbylang |
lang/triggerGeneral.sql |
test requires java.sql.DriverManager |
derbytools |
tools/ij4.sql |
ij test with no default driver or database |
derbytools |
tools/ij.sql |
test is specific to ij and tests bad/good urls |
derbytools |
tools/ij5.sql |
test uses ij.connection.<connection name> |
derbytools |
tools/ijConnName.sql |
test uses ij.connection.<connection name> |
storemore |
store/Beetle6038.java |
encryption not supported in Derby J2ME implementation |
storemore |
store/OnlineBackupTest3.java |
test uses procedure with server-side JDBC |
storemore |
store/BackupPathTests.java |
test uses procedure with server-side JDBC |
storerecovery |
store/LogChecksumSetup.java |
uses derby.subSubProtocol in URL |
store |
store/LogChecksumRecovery.java |
uses derby.subSubProtocol in URL |
store |
store/LogChecksumRecovery1.java |
uses derby.subSubProtocol in URL |
storeunit |
unit/T_RawStoreFactory.unit |
DERBY-654 |
i18nTest |
i18n/messageLocale.sql |
test requires java.sql.Driver |
i18nTest |
i18n/caseI_tr_TR.sql |
test requires java.sql.Driver |
The following table lists tests which can possibly be enabled to run with CDC.
Test Suite |
Test Name |
Reason |
derbylang |
lang/authorize.sql |
test uses server-side JDBC - exclude for now |
derbylang |
lang/connect.sql |
test uses ij 'driver' command - exclude for now |
derbylang |
lang/errorCode.sql |
test uses ij 'driver' command - exclude for now |
jdk14 |
jdbcapi/checkDataSource30.java |
This is based on checkDataSource test which is enabled |
derbylang |
lang/dcl.sql |
uses server-side JDBC - exclude for now |
derbylang |
lang/functions.sql |
test uses server-side JDBC - exclude for now - can be written as Java test |
derbytools |
tools/dblook_test.sql |
Possible rewrite using DataSource class |
derbytools |
tools/URLCheck.sql |
reason for exclusion not specfied |
derbytools |
tools/ij6.sql |
reason for exclusion not specfied |
jdbcapi |
jdbcapi/UpdateXXXTest.junit |
reason for exclusion not specfied |
JIRA issues
The following JIRA issues have been opened for this effort.
Jira Entry |
Comments |
Status |
Exclude tests which cannot be run in CDC |
Fixed |
|
Found by this effort |
Fixed |
|
Found by this effort |
Fixed |
|
Create a test wrapper to run tests using BigDecimal |
Fixed |
|
Found by this effort |
Fixed |
|
Doc task for DERBY-470 |
Fixed |
|
Found by this effort |
Fixed |
|
Opened to enable more tests which use ij |
Fixed |
|
Make show connections statement work in CDC |
Fixed |
|
Found by this effort |
Not started |
|
test junitTest/LangSuite cannot be run with J2ME |
Fixed |
Progress Table
The following table lists the svn revisions for fixes checked in for this effort:
Trunk Revision |
10.1 Revision |
Pass % |
Comment |
|
|
77.2% |
|
2005/10/25 |
80.75% |
Enhance ij connect statement to work in J2ME/CDC/FP |
|
2005/11/03 |
81.22% |
Make show connections statement work in CDC |
|
2005/11/08 |
81.22% |
Make show connections statement work in CDC |
|
2005/11/11 |
82.43% |
Enable tests in jdbcapi and derbylang suite |
|
2005/11/23 |
87.35% |
Bring store tests into J2ME/CDC Foundation testing |