org.apache.flink.runtime.fs.hdfs

Class HadoopRecoverableWriter

java.lang.Object

org.apache.flink.runtime.fs.hdfs.HadoopRecoverableWriter

All Implemented Interfaces:

org.apache.flink.core.fs.RecoverableWriter

@Internal
public class HadoopRecoverableWriter
extends Object
implements org.apache.flink.core.fs.RecoverableWriter

An implementation of the RecoverableWriter for Hadoop's file system abstraction.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.core.fs.RecoverableWriter

org.apache.flink.core.fs.RecoverableWriter.CommitRecoverable, org.apache.flink.core.fs.RecoverableWriter.ResumeRecoverable

Constructor Summary

Constructors

Constructor and Description
HadoopRecoverableWriter(org.apache.hadoop.fs.FileSystem fs) Creates a new Recoverable writer.

Method Summary

Modifier and Type	Method and Description
boolean	cleanupRecoverableState(org.apache.flink.core.fs.RecoverableWriter.ResumeRecoverable resumable)
org.apache.flink.core.io.SimpleVersionedSerializer<org.apache.flink.core.fs.RecoverableWriter.CommitRecoverable>	getCommitRecoverableSerializer()
org.apache.flink.core.io.SimpleVersionedSerializer<org.apache.flink.core.fs.RecoverableWriter.ResumeRecoverable>	getResumeRecoverableSerializer()
org.apache.flink.core.fs.RecoverableFsDataOutputStream	open(org.apache.flink.core.fs.Path filePath)
org.apache.flink.core.fs.RecoverableFsDataOutputStream	recover(org.apache.flink.core.fs.RecoverableWriter.ResumeRecoverable recoverable)
org.apache.flink.core.fs.RecoverableFsDataOutputStream.Committer	recoverForCommit(org.apache.flink.core.fs.RecoverableWriter.CommitRecoverable recoverable)
boolean	requiresCleanupOfRecoverableState()
boolean	supportsResume()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopRecoverableWriter

public HadoopRecoverableWriter(org.apache.hadoop.fs.FileSystem fs)

Creates a new Recoverable writer.

Parameters:

fs - The Hadoop file system on which the writer operates.

Method Detail

open

public org.apache.flink.core.fs.RecoverableFsDataOutputStream open(org.apache.flink.core.fs.Path filePath)
                                                            throws IOException

Specified by:

open in interface org.apache.flink.core.fs.RecoverableWriter

Throws:

IOException

recover

public org.apache.flink.core.fs.RecoverableFsDataOutputStream recover(org.apache.flink.core.fs.RecoverableWriter.ResumeRecoverable recoverable)
                                                               throws IOException

Specified by:

recover in interface org.apache.flink.core.fs.RecoverableWriter

Throws:

IOException

requiresCleanupOfRecoverableState

public boolean requiresCleanupOfRecoverableState()

Specified by:

requiresCleanupOfRecoverableState in interface org.apache.flink.core.fs.RecoverableWriter

cleanupRecoverableState

public boolean cleanupRecoverableState(org.apache.flink.core.fs.RecoverableWriter.ResumeRecoverable resumable)
                                throws IOException

Specified by:

cleanupRecoverableState in interface org.apache.flink.core.fs.RecoverableWriter

Throws:

IOException

recoverForCommit

public org.apache.flink.core.fs.RecoverableFsDataOutputStream.Committer recoverForCommit(org.apache.flink.core.fs.RecoverableWriter.CommitRecoverable recoverable)
                                                                                  throws IOException

Specified by:

recoverForCommit in interface org.apache.flink.core.fs.RecoverableWriter

Throws:

IOException

getCommitRecoverableSerializer

public org.apache.flink.core.io.SimpleVersionedSerializer<org.apache.flink.core.fs.RecoverableWriter.CommitRecoverable> getCommitRecoverableSerializer()

Specified by:

getCommitRecoverableSerializer in interface org.apache.flink.core.fs.RecoverableWriter

getResumeRecoverableSerializer

public org.apache.flink.core.io.SimpleVersionedSerializer<org.apache.flink.core.fs.RecoverableWriter.ResumeRecoverable> getResumeRecoverableSerializer()

Specified by:

getResumeRecoverableSerializer in interface org.apache.flink.core.fs.RecoverableWriter

supportsResume

public boolean supportsResume()

Specified by:

supportsResume in interface org.apache.flink.core.fs.RecoverableWriter