...
Use the following code:
Code Block language bash # Initialize virtual environment called "env" in ~/.virtualenvs or any other directory. (Consider using pyenv, to manage the python version as well as installed packages in your virtual environment) $ python3 -m venv ~/.virtualenvs/env # Activate virtual environment. $ . ~/.virtualenvs/env/bin/activate # Upgrade other tools. (Optional) pip install --upgrade pip pip install --upgrade setuptools # Install setup.py requirements. (env) $ pip install -r build-requirements.txt # Install Apache Beam package in editable mode. (env) $ pip install -e .[gcp,test]
For certain systems, particularly Macs with M1 chips, this installation method may not generate urns correctly. If running
python gen_protos.py
doesn't resolve the issue, consult https://github.com/apache/beam/issues/22742#issuecomment-1218216468 for further guidance.
...
Code Block | ||
---|---|---|
| ||
cd sdks/python/ pip install build && python setup.py -m build --sdist |
We will use the tarball built by this command in the --sdk_location
parameter.
...
- Click on a recent `Build python source distribution and wheels job` that ran successfully on the github.com/apache/beam master branch from this list.
- Click on List files on Google Cloud Storage Bucket on the right-side panel.
- Expand List file on Google Cloud Storage Bucket in the main panel.
- Locate and Download the ZIP file. For example,
apache-beam-2.4852.0.dev0.zip
fromtar.gz
from GCS. - It’s simplest to download the file using your browser by replacing the prefix “gs://” with “https://storage.googleapis.com/” . For example, https://storage.googleapis.com/beam-wheels-staging/master/02bf081d0e86f16395af415cebee2812620aff4b-207975627/apache-beam-2.25.0.dev0.zip
- Or follow these instructions to download using the
gsutil
command-line tool. Install the downloaded zip file. For example:
Code Block language bash title SimpleTest pip install apache-beam-2.4852.0.dev0.ziptar.gz # Or, if you need extra dependencies: pip install apache-beam-2.4852.0.dev0.tar.zipgz[aws,gcp]
- When you run your Beam pipeline, pass in the
--sdk_location
flag pointed at the same ZIP file.Code Block language bash title SimpleTest --sdk_location=apache-beam-2.2552.0.dev0.ziptar.gz
How to update dependencies that are installed in Python container images
...