diff options
author | Paul Rogers <progers@cloudera.com> | 2019-03-03 17:55:22 -0800 |
---|---|---|
committer | Vitalii Diravka <vitalii.diravka@gmail.com> | 2019-03-05 16:29:42 +0200 |
commit | 7e3b45967dbb97da18ba49a2fa6a67a48e33b092 (patch) | |
tree | 3f46493d29571675fbe3d4e839cbb7f15311fa08 /exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/ScanOperatorEvents.java | |
parent | 2c3e2de2f94fd3f21a11c22b7944b94953e4f397 (diff) |
DRILL-7074: Scan framework fixes and enhancements
Roll-up of fixes an enhancements that emerged from the effort to host the CSV reader on the new framework.
closes #1676
Diffstat (limited to 'exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/ScanOperatorEvents.java')
-rw-r--r-- | exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/ScanOperatorEvents.java | 42 |
1 files changed, 32 insertions, 10 deletions
diff --git a/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/ScanOperatorEvents.java b/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/ScanOperatorEvents.java index 9e174146f..04b2c7eb3 100644 --- a/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/ScanOperatorEvents.java +++ b/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/ScanOperatorEvents.java @@ -20,15 +20,23 @@ package org.apache.drill.exec.physical.impl.scan; import org.apache.drill.exec.ops.OperatorContext; /** - * Interface to the set of readers, and reader schema, that the - * scan operator manages. The reader factory creates and returns - * the readers to use for the scan, as determined by the specific - * physical plan. The reader factory also - * translates from the select list provided - * in the physical plan to the actual columns returned from the - * scan operator. The translation is reader-specific; this - * interface allows the scan operator to trigger various - * lifecycle events. + * Interface to the set of readers, and reader schema, that the scan operator + * manages. The reader factory creates and returns the readers to use for the + * scan, as determined by the specific physical plan. The reader factory also + * translates from the select list provided in the physical plan to the actual + * columns returned from the scan operator. The translation is reader-specific; + * this interface allows the scan operator to trigger various lifecycle events. + * <p> + * This interface decouples the scan implementation from the generic tasks + * needed to implement Drill's Volcano iterator protocol for operators, and + * Drill's schema and batch semantics. A scan implementation need only + * implement this interface to add plugin-specific scan behavior. + * <p> + * While this interface allows a wide variety of implementations, the intent is + * that most actual scanners will use the "managed" framework that handles the + * routine projection, vector management and other tasks that tend to be common + * across scanners. See {@link ScanSchemaOrchestrator} for the managed + * framework. */ public interface ScanOperatorEvents { @@ -46,11 +54,25 @@ public interface ScanOperatorEvents { void bind(OperatorContext context); + /** + * A scanner typically readers multiple data sources (such as files or + * file blocks.) A batch reader handles each read. This method returns + * the next reader in whatever sequence that this scan defines. + * <p> + * The preferred implementation is to create each batch reader in this + * call to minimize resource usage. Production queries may read + * thousands of files or blocks, so incremental reader creation can be + * far more efficient than creating readers at the start of the scan. + * + * @return a batch reader for one of the scan elements within the + * scan physical plan for this scan operator + */ + RowBatchReader nextReader(); /** * Called when the scan operator itself is closed. Indicates that no more - * readers are available (or will be opened). + * readers are available. */ void close(); |