...
When the CooldownType of a tablet is inconsistent with the status reported by BE, the CooldownType status needs to be synchronized first (first turn off the upload of the upload copy of the file being uploaded and set it to USE_REMOTE_DATA; then set the new upload copy status to UPLOAD_DATA). When the copy replica of a tablet Tablet is abnormal (the CooldownType of all copies replicas is USE_REMOTE_DATA), the tablet with the CooldownType specified as UPLOAD_DATA is abnormal, and the tablet with the CooldownType specified as UPLOAD_DATA has not reported the heartbeat (lastUpdateTime) after the timeout period, it is determined that the tablet needs to reselect the Upload copy and set the CooldownType to UPLOAD_ DATA。it is determined that the Tablet needs to re select the UPLOAD replica and set the CooldownType to Upload_DATA.
When the CooldownType needs to be issued, the FE side needs to send a request TPushCooldownConfReq, send the status to BE, and modify the UPLOADING file of the specified TABLET on S3_ REPLICA_ ID, the file content is that the current CooldownType is UPLOAD_ REPLICA of the copy of DATA_ ID。
The CooldownType on the FE side can only be accessed from the USE_ REMOTE_ DATA changes to UPLOAD_ DATA，UPLOAD_ Generally, the DATA node does not need to be changed unless BE times out. When BE times out, the FE side will automatically process the TABLET migration. The newly migrated TABLET is defaulted to USE_ REMOTE_ DATA, UPLOAD will be re specified_ DATA node. Ensure that two CooldownTypes are not UPLOAD at the same time_ Node of DATA.
...
The CooldownType status of each tablet on the BE side is USE by default_ REMOTE_ DATA, that is, do not upload. Only when FE sends a request to set CooldownType to UPLOAD_ Only DATA updates its status and sets it to upload.If the heartbeat time reported by the Tablet last exceeds the timeout
threshold mentioned above, according to the timeout processing mechanism of the FE end, the current Tablet has been set When CooldownType is UPLOAD_ During DATA, check UPLOADING on S3_ REPLICA_ ID, if UPLOADING_ REPLICA_ If the ID is not the current REPLICA, set the local CooldownType to USE_ REMOTE_ The possibility of DATA. At this time, set CooldownType to USE_ REMOTE_ DATA, stop uploading. Later, wait for FE to re initiate the request and redistribute CooldownType to BE.In the process of uploading files on the BE side, if the operation takes too long, the timeout threshold should also be checked regularly. If timeout is found, the current upload request should be stopped immediately and the uploaded files should be deleted. (If the remote TabletMeta has been updated, it indicates that the operation has been completed, and the file does not need to be deleted)DATA, no longer upload. If UPLOADING_ REPLICA_ The ID is the current REPLICA, upload the file, and regenerate the remote META file on S3. The META file name needs to contain the current REPLICA_ ID to prevent write conflicts with other replicas.
When CooldownType status is USE_ REMOTE_ When DATA, obtain the UPLOADING on S3_ REPLICA_ ID, according to REPLICA_ ID to find the corresponding remote META file. Get the META file and merge it with the local META. If the VERSION corresponding to the local ROWSET is
UPLOAD_ DATA is a super operation with high risk. It is necessary to ensure that only one copy is performing upload requests at the same time. Therefore, some real-time performance can be sacrificed to reduce the probability of this risk.
...
Because different copies of the same batch of imported data under the same tablet rowset_ The IDs are different. When uploading a segment, its corresponding TabletMeta should be uploaded for reference by other copies of the tablet. This information only needs to include the Upload segment, not the local segment.
When in coolownEach time CooldownType changes, the first launch of coolown will trigger metadata synchronization, first read the TabletMeta file under in the remote cluster directory, check the existing data range on the remote cluster according to it (which segments VERSIONS have been uploaded) on the remote cluster according to it, and compare it with , pull down the ROWSET metadata of the specified VESION, and replace the corresponding VERSION fragment in the local TabletMeta. If the remote TabletMeta of the local record is different, merge the remote TabletMeta.
According to CooldownType status, there are two operations:
If CooldownType is UPLOAD_ DATA, indicating that the current tablet needs to be uploaded. Upload the subsequent segments according to the data range on the cluster, and then update the remote TabletMeta. During the upload process, the CooldownType should also be checked regularly to prevent the CooldownType update from not being found because the upload time is too long.
In coolown, check UPLOADING on S3_ REPLICA_ ID, if UPLOADING_ REPLICA_ If the ID is not the current REPLICA, set the local CooldownType to USE_ REMOTE_ DATA, no longer upload. If UPLOADING_ REPLICA_ The ID is the current REPLICA, upload the file, and regenerate the remote META file on S3.
During the whole process of uploading files, the uploaded files are all your own files and will not conflict with other copies. In extreme cases, two copies are being uploaded at the same time, because UPLOADING_ REPLICA_ If the ID file exists, only the data of one copy file will be usedIf CooldownType is USE_ REMOTE_ DATA, indicating that it is not necessary to upload files. Skip the upload step and directly delete the local redundant segments after merging the TabletMeta.
Note: In order to prevent inconsistency between different replicas of the local Rowset, which will result in the inability to de duplicate the cold data cannot be de duplicated with the cold data on the remote cluster, the . The cold data compression operation only processes the segments on the remote cluster, not the local segmentswill not be compressed temporarily.
BE in USE_ REMOTE_ DATA type
...