FlinkStreamingPipelineTranslator
1 所有的可用Translator:
// here you can find all the available translators.
static {
TRANSLATORS.put(Read.Bounded.class,new BoundedReadSourceTranslator());
TRANSLATORS.put(Read.Unbounded.class,new UnboundedReadSourceTranslator());
TRANSLATORS.put(TextIO.Write.Bound.class,new TextIOWriteBoundStreamingTranslator());
TRANSLATORS.put(ParDo.MultIoUtput.class,new ParDoStreamingTranslator());
TRANSLATORS.put(
SplittableParDo.ProcessElements.class,new SplittableProcessElementsStreamingTranslator());
TRANSLATORS.put(
SplittableParDo.GBKIntoKeyedWorkItems.class,new GBKIntoKeyedWorkItemsTranslator());
TRANSLATORS.put(Window.Assign.class,new WindowAssignTranslator());
TRANSLATORS.put(Flatten.PCollections.class,new FlattenPCollectionTranslator());
TRANSLATORS.put(
FlinkStreamingViewOverrides.CreateFlinkPCollectionView.class,new CreateViewStreamingTranslator());
TRANSLATORS.put(Reshuffle.class,new ReshuffleTranslatorStreaming());
TRANSLATORS.put(GroupByKey.class,new GroupByKeyTranslator());
TRANSLATORS.put(Combine.PerKey.class,new CombinePerKeyTranslator());
}
2 applyStreamingTransform:
基本逻辑:将PTransform关联到streamingContext
// create the applied PTransform on the streamingContext streamingContext.setCurrentTransform(node.toAppliedPTransform());
typedTranslator.translateNode(typedTransform,streamingContext);
12种类型的translator多态实现了StreamTransformTranslator.translateNode(T transform,FlinkStreamingTranslationContext context);
方法。