...
| Code Block |
|---|
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>${shading-artifact.name}</artifactId>
<version>${project.version}</version>
</dependency> |
NOTE: It is very important not to put guava, protobuf, netty and asm (== all shaded hadoop dependencies) into the dependencyManagement section of the parent pom. If they are there, they'll influence the version we'll pack into the shaded hadoop jar!
External Shading: Guava, ASM
External shading hides some of Flink's dependencies to Flink users. This allows users to use whatever version of Guava or ASM they want.
The external shading is controlled in the flink-parent pom, in the maven-shade-plugin.
As per status of the pull request mentioned above, a project depending on:
Will see the following Flink dependencies:
| Code Block |
|---|
[INFO] --- maven-dependency-plugin:2.8:list (default-cli) @ debug-quickstart ---
[INFO]
[INFO] The following files have been resolved:
[INFO] org.objenesis:objenesis:jar:2.1:compile
[INFO] com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] com.twitter:bijection-core_2.10:jar:0.7.2:compile
[INFO] com.rabbitmq:amqp-client:jar:3.3.1:compile
[INFO] org.apache.flink:flink-compiler:jar:0.9-SNAPSHOT:compile
[INFO] com.typesafe.akka:akka-remote_2.10:jar:2.3.7:compile
[INFO] commons-fileupload:commons-fileupload:jar:1.3.1:compile
[INFO] com.google.code.simple-spring-memcached:spymemcached:jar:2.8.4:compile
[INFO] org.uncommons.maths:uncommons-maths:jar:1.2.2a:compile
[INFO] org.apache.flink:flink-streaming-connectors:jar:0.9-SNAPSHOT:compile
[INFO] stax:stax-api:jar:1.0.1:compile
[INFO] org.apache.flink:flink-streaming-core:jar:0.9-SNAPSHOT:compile
[INFO] org.tukaani:xz:jar:1.0:compile
[INFO] org.apache.thrift:libthrift:jar:0.6.1:compile
[INFO] commons-logging:commons-logging:jar:1.1.1:compile
[INFO] com.twitter:chill-java:jar:0.5.2:compile
[INFO] redis.clients:jedis:jar:2.4.2:compile
[INFO] org.apache.httpcomponents:httpcore:jar:4.2.4:compile
[INFO] com.esotericsoftware.kryo:kryo:jar:2.24.0:compile
[INFO] org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] de.javakaffee:kryo-serializers:jar:0.27:compile
[INFO] org.slf4j:slf4j-api:jar:1.7.7:compile
[INFO] org.apache.commons:commons-pool2:jar:2.0:compile
[INFO] org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] jline:jline:jar:0.9.94:compile
[INFO] com.twitter:chill-thrift:jar:0.5.2:compile
[INFO] log4j:log4j:jar:1.2.17:compile
[INFO] com.google.guava:guava:jar:11.0.2:compile
[INFO] com.amazonaws:aws-java-sdk:jar:1.8.1:compile
[INFO] com.twitter:chill-protobuf:jar:0.5.2:compile
[INFO] com.twitter:bijection-avro_2.10:jar:0.7.2:compile
[INFO] com.twitter:chill_2.10:jar:0.5.2:compile
[INFO] com.thoughtworks.paranamer:paranamer:jar:2.3:compile
[INFO] org.apache.httpcomponents:httpclient:jar:4.2.5:compile
[INFO] commons-lang:commons-lang:jar:2.5:compile
[INFO] joda-time:joda-time:jar:2.5:compile
[INFO] com.fasterxml.jackson.core:jackson-databind:jar:2.1.1:compile
[INFO] com.twitter:hbc-core:jar:2.2.0:compile
[INFO] com.twitter:chill-bijection_2.10:jar:0.5.2:compile
[INFO] org.apache.kafka:kafka_2.10:jar:0.8.2.0:compile
[INFO] org.apache.flink:flink-shaded-include-yarn:jar:0.9-SNAPSHOT:compile
[INFO] com.fasterxml.jackson.core:jackson-core:jar:2.1.1:compile
[INFO] org.eclipse.jetty:jetty-continuation:jar:8.0.0.M1:compile
[INFO] org.eclipse.jetty:jetty-util:jar:8.0.0.M1:compile
[INFO] io.netty:netty-all:jar:4.0.24.Final:compile
[INFO] com.yammer.metrics:metrics-core:jar:2.2.0:compile
[INFO] commons-cli:commons-cli:jar:1.2:compile
[INFO] com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] org.apache.zookeeper:zookeeper:jar:3.4.6:compile
[INFO] com.fasterxml.jackson.core:jackson-annotations:jar:2.1.1:compile
[INFO] org.slf4j:slf4j-log4j12:jar:1.7.7:compile
[INFO] org.eclipse.jetty:jetty-servlet:jar:8.0.0.M1:compile
[INFO] org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] com.typesafe.akka:akka-slf4j_2.10:jar:2.3.7:compile
[INFO] org.xerial.snappy:snappy-java:jar:1.0.5:compile
[INFO] org.apache.flink:flink-java:jar:0.9-SNAPSHOT:compile
[INFO] org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] org.apache.commons:commons-lang3:jar:3.3.2:compile
[INFO] org.apache.flume:flume-ng-sdk:jar:1.5.0:compile
[INFO] com.esotericsoftware.minlog:minlog:jar:1.2:compile
[INFO] org.apache.flink:flink-runtime:jar:0.9-SNAPSHOT:compile
[INFO] org.eclipse.jetty:jetty-server:jar:8.0.0.M1:compile
[INFO] org.mortbay.jetty:servlet-api:jar:3.0.20100224:compile
[INFO] commons-collections:commons-collections:jar:3.2.1:compile
[INFO] io.netty:netty:jar:3.5.12.Final:compile
[INFO] com.101tec:zkclient:jar:0.3:compile
[INFO] org.apache.kafka:kafka-clients:jar:0.8.2.0:compile
[INFO] org.apache.flink:flink-core:jar:0.9-SNAPSHOT:compile
[INFO] org.apache.mina:mina-core:jar:2.0.4:compile
[INFO] com.github.scopt:scopt_2.10:jar:3.2.0:compile
[INFO] com.typesafe:config:jar:1.2.1:compile
[INFO] org.apache.commons:commons-math:jar:2.2:compile
[INFO] junit:junit:jar:4.4:compile
[INFO] org.apache.sling:org.apache.sling.commons.json:jar:2.0.6:compile
[INFO] org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] commons-io:commons-io:jar:2.4:compile
[INFO] net.jpountz.lz4:lz4:jar:1.2.0:compile
[INFO] org.eclipse.jetty:jetty-io:jar:8.0.0.M1:compile
[INFO] org.apache.flink:flink-clients:jar:0.9-SNAPSHOT:compile
[INFO] com.twitter:joauth:jar:6.0.2:compile
[INFO] org.apache.avro:avro-ipc:jar:1.7.3:compile
[INFO] org.apache.avro:avro:jar:1.7.6:compile
[INFO] com.typesafe.akka:akka-actor_2.10:jar:2.3.7:compile
[INFO] org.apache.flume:flume-ng-core:jar:1.5.0:compile
[INFO] org.apache.flume:flume-ng-configuration:jar:1.5.0:compile
[INFO] org.eclipse.jetty:jetty-security:jar:8.0.0.M1:compile
[INFO] commons-codec:commons-codec:jar:1.3:compile
[INFO] org.eclipse.jetty:jetty-http:jar:8.0.0.M1:compile
[INFO] com.twitter:chill-avro_2.10:jar:0.5.2:compile
[INFO] org.scala-lang:scala-library:jar:2.10.4:compile
|