| Panel | ||||||
|---|---|---|---|---|---|---|
| ||||||
|
...
The Open Climate Workbench (OCW) project has basic unit test coverage in the form of a test suite written using the nose framework. In order to improve test coverage, functionality, and quality of tests the test framework should be updated. OCW would greatly benefit from a continuous integration (CI) infrastructure and automated test environment with a focus on large scale integration testing of climate model analysis. The current testing makes makes the process of new code integration tedious and manual with potential regressions going unnoticed, this can lead to bugs and backwards backward incompatibility. This GSoC project proposes to build an improved test framework (smoke tests and unit tests) and set up continuous integration tools.
...
The key deliverable of this project involves improving the existing unit tests before designing designing and developing a brand new smoke test suite which will then enable CI testing for OCW. In the second half of the project, the continuous integration infrastructure will be built and fully automated. This involves configuring the dedicated build clusters at Apache Builds (and also at Travis-CI). The continuous integration server would automatically build and test the source code for each pull request that is submitted and post the results. All documentations related to the project will also be created. If time permits, I would also like to improve the overall project by fixing the numerous issues the project currently has.
...
This project will result in a robust and effective test suit for the OCW project along with a working continuous integration environments environment that tests the source code and provide provides useful results. Also, there will be documentations for the new additions to OCW.Objectives:
...
During the community bonding period, I will solicit the OCW community for feedback on designing of the smoke tests and what the current unit tests lack. I will also try to develop a prototype for the tests in this period which would be useful during the coding period. The OCW project has some automated installation scripts but they have some issues. During this period I’ll also focus on fixing these issues so that the continuous integration environment can be set up effectively.
...
By the end of week 3, the unit test framework should be completed and coding for the smoke tests can be started. The weeks 4, 5 and 6 will be used to develop the smoke testing framework for the OCW project. The design and implementation (prototype) of this framework will be done during the community bonding period.
July 15, 2016 - July 30, 2016 (Week 7 - 8)
The aim of weeks 7 and 8 is to integrate the project to into a continuous integration environment. Build(s) job(s) will be established on builds.apache.org to run the new large scale integration tests. Jenkins is the main integration environment which will be configured to automatically execute the test and build the project.
...
This is the buffer period in case things don’t turn out as planned. All code that remains incomplete or has bugs will be fixed in this period.
Every sunday Sunday I will write documentation for all code that is created along with the weekly progress report.
...
Week 1 (23rd April - 30th April)
Previous Action Items: None
Weekly Activity:Week 1 was used to find where the existing unit tests lack and which modules need to be tested. A test build has been set up at Codecov to generate test coverage reports which will be used for further analysis and development of tests. Also, Jenkins build (which would always fail) has been fixed partially and now It completes successfully.
Issues found - CLIMATE-785, CLIMATE-786, CLIMATE-787
Issues fixed - CLIMATE-785, CLIMATE-786
Next Week's Agenda: Study and analyse analyze dataset.py and test_dataset.py file in the ocw module.Mentor's Comments:Community bonding and engagement has been good. Lots of activity
Week 2 (1st May - 7th May)
Previous Action Items: Fix CLIMATE-787
Weekly Activity:
The Open Climate Workbench module and the test suite were studied further in this week, especially the dataset.py and test_dataset.py files.Multiple issues were found in these files which will be addressed in the coming weeks.
Issues found - CLIMATE-788, CLIMATE-788
Issues fixed - CLIMATE-789
Next Week's Agenda: Study and analyse analyze dataset_processor.py and test_dataset_processor.py file in the ocw module.
Mentor's Comments: Community bonding and engagement has been good. Lots of activity
Week 3 (8th May - 14th May)
Previous Action Items: Fix CLIMATE-787 and CLIMATE-788
The study of test suite was continued in this week. The dataset_processor.py file and test_dataset_processor.py file were studied specifically. A prototype for the functions that are not covered through by the tests was developed. Also, the existing test files did not comply with PEP8. All files in the test directory have been updated to comply with PEP8 guidelines.
Weekly Activity:
Issues found - CLIMATE-791, CLIMATE-792
Issues fixed - CLIMATE-791, CLIMATE-792Next Week's Agenda: Study and analyse analyze evaluations.py and test_evaluations.py file in the ocw module.
Mentor's Comments: PR's were sent in for a number of issues. Ibrahim has been in contact with me and things are going well. He also logged an issue with INFRA regarding pre-commit build hooks for OCW, Jenkins + Github integration.
Week 4 (15th May - 21th May)
Previous Action Items: Fix CLIMATE-787 and CLIMATE-788
Weekly Activity:The aim of the week 4 was to study the evaluation.py file and tests related to it, which was achieved. I managed to trace out which tests need to be updated and which functions are not covered currently. I couldn't contribute much during this week as I was having my end sessional examination. I've conveyed the same to my mentor.
Issues found -
Issues fixed -Next Week's Agenda: Add tests to test_dataset_processor.py file to improve test coverage.
Mentor's Comments: I have been working with Ibrahim Jarif getting him to commit directly to the master canonical source repository hosted at Apache. I am happy with progress so far but would like to see Ibrahim Jarif start thinking about larger scale integration tests. He should also be following up with Infrastructure on getting the Jenkins build issues sorted out as well as documenting the CI infrastructure on the Climate wiki.
Week 5 (22nd May - 28th May)
Previous Action Items: Fix CLIMATE-787 and CLIMATE-788
Weekly Activity:
I managed to add new test cases for dataset processor module. The tests coverage has been increased. Some more tests need to be added which will be completed in the following week. The build at Apache Builds has been fixed. The build is triggered every time a PR is submitted to the climate repository. The builds are failing due to some permission issue with conda package manager. I've been following up with the Apache Infra team for support and the build should be fixed in the next week. Also coveralls integration as been added which keeps track of code coverage.Issues found - CLIMATE-793, CLIMATE-794, CLIMATE-795
Issues fixed - CLIMATE-788, CLIMATE-793, CLIMATE-794Next Week's Agenda: Complete remaining dataset processor tests and add tests for evaluation and dataset module.
Mentor's Comments: A good solid week of work. I am happy with progress.
Week 6 (29th May - 4th June)
Previous Action Items: Fix CLIMATE-787 and CLIMATE-795
Weekly Activity:
Completed the tests related to dataset processor module, evaluation module, and the dataset module.
Improved dataset processor tests coverage from 44.66% to 62%, evaluation tests coverage from 87.13% to 94%, dataset module tests coverage from 84.21% to 100%. Some more tests will be added to these modules after discussing with the mentor. Multiple bugs were fixed in these modules while developing the tests.Issues found - CLIMATE-796, CLIMATE-799, CLIMATE-800, CLIMATE-801, CLIMATE-802
Issues fixed - CLIMATE-795, CLIMATE-796, CLIMATE-799, CLIMATE-800, CLIMATE-801, CLIMATE-802Next Week's Agenda: Add tests for utils module and local module.
Mentor's Comments:
Week 7 (5th June - 11th June)
Previous Action Items: Fix CLIMATE-787
Weekly Activity:
Completed the tests related to utils module and local module. More tests will be added for the local module after CLIMATE-787 has been resolved.
Improved utils module tests coverage from 69.93% to 100%, local module tests coverage from 29.84% to 51%.Issues found - CLIMATE-803, CLIMATE-804, CLIMATE-805, CLIMATE-806
Issues fixed - CLIMATE-804, CLIMATE-805, CLIMATE-806Next Week's Agenda: Develop Smoke tests prototype.
Mentor's Comments:
Week 8 (12th June - 18th June)
Previous Action Items: Fix CLIMATE-787
Weekly Activity:
The aim of this week was to develop smoke tests prototype which unfortunately remains pending. However, I managed to fix some issues and update ocw module code base. I added integration for landscape.io and requires.io. Also, all the files in the ocw module now comply with PEP8. In the next week, I'll complete the smoke tests prototype.Issues found - CLIMATE-807, CLIMATE-808, CLIMATE-809, CLIMATE-811, CLIMATE-812, CLIMATE-813, CLIMATE-814, CLIMATE-815, CLIMATE-816
Issues fixed - CLIMATE-807, CLIMATE-808, CLIMATE-809, CLIMATE-811, CLIMATE-812, CLIMATE-813, CLIMATE-814, CLIMATE-815, CLIMATE-816Next Week's Agenda: Develop Smoke tests prototype.
Mentor's Comments
Midterm Report
Final Report
...
Week 9 - Till Completion (From 19th June)
Activity:
The smoke test prototype was developed in this period. Along with it, the remaining unit tests were improved and the continuous integration infrastructure was developed. The reports related to a few weeks are missing as the project was completed ahead of the expected timeline.Mentor's Comments
Final Report
The project was successfully completed with the following outcomes:
- An improved set of Unit Tests
- Newly developed Smoke Tests and
- A working Continuous Integration Infrastructure.