This Wiki page is to coordinate the testing efforts in the community. For the community testing, the feature authors are ideally not testing their own features. They are just providing minimal pointers to the docs or configuration parameters, so that we can test the usability of the new feature as well.

Release Testing Status

Current Release Candidate: RC2 (December 1st, 2020)

Binaries: https://dist.apache.org/repos/dist/dev/flink/flink-1.12.0-rc2/

Maven Staging: https://repository.apache.org/content/repositories/orgapacheflink-1403/

Flink 1.12 Burndown (blockers and critical bugs): https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=406&projectKey=FLINK

Currently known major bugs:

  • New Kafka Source is not working ( FLINK-20157 - Getting issue details... STATUS ) - present in RC2, the Kafka source will not be announced as a new feature.

Testing Topics

When testing a feature, consider the following aspects:

  • Is the documentation easy to understand
  • Are the error messages, log messages, APIs etc. easy to understand
  • Is the feature working as expected under normal conditions
  • Is the feature working / failing as expected with invalid input, induced errors etc.


Topic
(+ Feature Ticket)

Tested by

(Please state your name, and the RC you've used)

Testing Ticket

Issues foundNotes

K8s High Availability Service

FLINK-12884 - Getting issue details... STATUS

FLINK-20113 - Getting issue details... STATUS



Kafka Source based on the new Source API  FLINK-18323 - Getting issue details... STATUS

FLINK-20114 - Getting issue details... STATUS



Batch execution for the DataStream API  FLINK-19316 - Getting issue details... STATUS / FLINK-19268 - Getting issue details... STATUS / FLINK-19758 - Getting issue details... STATUS

FLINK-20115 - Getting issue details... STATUS


Note to testers: The three issues can really only be tested in combination. FLINK-19316 is done but missing documentation.

  • Write an example that uses a (new) FileSource, a (new) FileSink, some random transformations
  • Run the example in BATCH mode
  • How ergonomic is the API/configuration?
  • Are there any weird log messages/exceptions in the JM/TM logs
  • Maybe try sth that doesn't work on BATCH execution, such as iterations/feedback edges.

Intra-Slot Managed Memory Sharing

FLINK-19177 - Getting issue details... STATUS

FLINK-20116 - Getting issue details... STATUS



Flink on Mesos Deployment

FLINK-18722 - Getting issue details... STATUS

Matthias

FLINK-20117 - Getting issue details... STATUS


This is not a new feature. We have touched the Mesos deployment during the ResourceManager refactor. Would be helpful verify the deployment with manual tests.

New File Source API - Continuous Streaming Execution

FLINK-19161 - Getting issue details... STATUS

FLINK-20118 - Getting issue details... STATUS



FLIP-132 Temporal Table DDL and Temporal Table Join

FLINK-16824 - Getting issue details... STATUS

FLINK-20119 - Getting issue details... STATUS



FLIP-107: Handling of metadata in SQL connectors

FLINK-15869 - Getting issue details... STATUS

FLINK-20120 - Getting issue details... STATUS



FLIP-149: Introduce the upsert-kafka Connector

FLINK-19857 - Getting issue details... STATUS

FLINK-20121 - Getting issue details... STATUS



Introduce multi-input operator

FLINK-19621 - Getting issue details... STATUS



Note to testers: check if log and Web UI is showing meaningful names of tasks with chained sources.

Temporal Join on Hive table

FLINK-19644 - Getting issue details... STATUS




Small file compaction

FLINK-19345 - Getting issue details... STATUS

FLINK-20122 - Getting issue details... STATUS



Pandas UDAF in Python Table API
FLINK-19169 - Getting issue details... STATUS

FLINK-20133 - Getting issue details... STATUS



Python UDAF in Python Table API
FLINK-19185 - Getting issue details... STATUS

FLINK-20134 - Getting issue details... STATUS



Python DataStream API
FLINK-18761 - Getting issue details... STATUS

FLINK-20135 - Getting issue details... STATUS



Python workers resource management
FLINK-19181 - Getting issue details... STATUS

FLINK-20136 - Getting issue details... STATUS



Native support of PyFlink on Kubernetes

FLINK-17480 - Getting issue details... STATUS

FLINK-20123 - Getting issue details... STATUS



Support collect in the Python Table API
FLINK-18415 - Getting issue details... STATUS

FLINK-20136 - Getting issue details... STATUS



Non-blocking job submission

FLINK-16866 - Getting issue details... STATUS

FLINK-20126 - Getting issue details... STATUS


Test proper error reporting in case JobManager initialization hangs

FLIP-119: Pipelined region scheduler

FLINK-16430 - Getting issue details... STATUS

Robert (in progress)

FLINK-20124 - Getting issue details... STATUS


high parallelism; many failures; many regions; large cluster;

Displaying memory configuration in the web ui

FLINK-19661 - Getting issue details... STATUS

FLINK-14431 - Getting issue details... STATUS

FLINK-20125 - Getting issue details... STATUS


The following questions should be answered for the different deployment modes (application cluster, per-Job cluster, session cluster) and deployment backends (Standalone/Local, Kubernetes, Yarn, Mesos):

  • Does the TaskManager metrics page (path: /#/task-manager/<taskmanager-id>/metrics) looks as proposed in FLIP-102 (minor improvements where introduced in  FLINK-19897 - Getting issue details... STATUS )? It helps to have a job running to see whether the metrics in the right column are actually dynamically changing.
  • Check whether all 8 newly introduced metrics are exposed through the TaskManager's metric endpoint:

    Metrics path
    http://localhost:8081/taskmanagers/<taskmanager-id>/metrics?get=Status.Flink.Memory.Managed.Total,Status.Flink.Memory.Managed.Used,Status.JVM.Memory.Metaspace.Committed,Status.JVM.Memory.Metaspace.Used,Status.JVM.Memory.Metaspace.Total,Status.Shuffle.Netty.UsedMemory,Status.Shuffle.Netty.AvailableMemory,Status.Shuffle.Netty.TotalMemory 
  • Does the JobManager metrics page (path: /#/job-manager/metrics) as proposed in FLIP-104?
  • Does the Cluster configuration endpoint include /#/job-manager/config include memory-related parameters like jobmanager.memory.heap.sizejobmanager.memory.jvm-metaspace.size, or jobmanager.memory.off-heap.size? Do they match configuration parameters possibly set in the config file?

Improved unaligned checkpointing times under backpressure

FLINK-16972 - Getting issue details... STATUS

Arvid

FLINK-20127 - Getting issue details... STATUS



Test SQL Hive integration

FLINK-20151 - Getting issue details... STATUS


Test how easy it is to use Flink's Hive integration. In particular pay attention to

  • Dependency management is easy (no unclear dependencies)
  • Configuration works smoothly (no relative file paths in configuration)
  • No labels