See the Go SDK Developer README at the sdks/go root for getting started with working on the Go SDK. In particular, the SDK is currently opinionated about where the beam repo needs to be copied in order for Go to use your SDK changes rather than it's own cached copy.
Development Recommendations
- Follow the instructions in Go SDK Developer README at the sdks/go root for getting started with working on the Go SDK, in particular with how the Beam repo needs to be nested in the GOPATH.
- Failure to do so will mean you won't be executing your changes to the SDK.
- Use the Python Portable Runner for end to end testing and validation of pipelines
- Follow the instructions in Python Tips to have a working python SDK environment.
- Ensure you're in the python virtual env.
- From the sdks/python directory, start a stand alone Python Portable Runner Job Service
python -m apache_beam.runners.portability.local_job_service_main --port=8099
- Submit jobs with the flags :
--runner=universal --endpoint=localhost:8099 --environment_type=LOOPBACK
- If this seems like too many steps, consider making the Python Portable runner more easily accessible to the Go SDK – BEAM-11077
- Do not use the "Go Direct" runner. It's not much more than a toy suitable for limited basic batch testing. It's a separate task to make it more generally useful – BEAM-11076