Code
The code for the extensions can be found on GitHub.
Prerequisites
Install:
- Java
- Maven
- Docker + Docker-Compose
- Here we describe the setup in IntelliJ (we do provide run configuration files to ease the configuration setup), but you can use an IDE of your choice
Development Environment
In this setup we will start extensions such as pipeline elements in your IDE. The backend, UI and all the other services run in docker.
Set-Up StreamPipes Services
Clone Installer [Link]
Prepare Installer# SSH git clone git@github.com:apache/incubator-streampipes-installer.git # HTTPS git clone https://github.com/apache/incubator-streampipes-installer.git # switch to development branch git checkout dev
Open CLI directory
Prepare Installercd incubator-streampipes-installer cd cli
Configure StreamPipes for backend development
Prepare Installer./streampipes env --set pipeline-element # Updates the file 'pipeline-element'. All StreamPipes services required for development are selected
Start third party services (e.g. CouchDB, Kafka, ... in docker)
Prepare Installer./streampipes up -d
Set-Up repository
Clone StreamPipes [Link]
Prepare Installer# SSH git clone git@github.com:apache/incubator-streampipes-extensions.git # HTTPS git clone https://github.com/apache/incubator-streampipes-extensions.git
Open the project in your IDE
Set-Up for Extensions (Pipeline Elements, Connect Adapter)
We provide an environment file for each service in the folder development. This contains the environment variables that must be set to run the service.
For IntelliJ users: we ship run configurations with the repo located at .idea/runConfigurations
For other IDE users: You need to manually set environment variables for the corresponding service according to the specification found in the run configurations for IntelliJ
Example walkthrough for pipeline-elements-jvm-all in IntelliJ:
This is a service that wraps all pipeline elements written with the standard Java wrapper, hence the the notion of "jvm-all".
- You should automatically have a run configuration "all-pipeline-elements-jvm" as we ship in with the repository
- (Optional) Modifiy environment variables depending on your OS (this is due to the dev setup of partly IDE and partly Docker, we need the local services to be reachable from within the Docker network)
- For Windows/Mac user: you are good to go
- For Linux user: You must change the environment variables stating host.docker.internal to you local machine's IP
- Run the service
- Navigate to "http://localhost:7023/" to see if the backend is running. You should see something similar to the following: