Apache Beam使用早期触发器触发窗格的次数

时间:2022-11-20 19:17:49

In a streaming beam pipeline, a trigger is set to be

在流式波束流水线中,触发器被设置为

Window.into(FixedWindows.of(Duration.standardHours(1)))
              .triggering(AfterWatermark
                            .pastEndOfWindow()
                            .withEarlyFirings(AfterProcessingTime
                                    .pastFirstElementInPane()
                                    .plusDelayOf(Duration.standardMinutes(15))))
              .withAllowedLateness(Duration.standardHours(1))
              .accumulatingFiredPanes())
  1. If there's no new data between the early firing (15 minutes after the first element of the current window) and the watermark, will there be another firing at the end of the watermark?

    如果在早期射击(当前窗口的第一个元素之后15分钟)和水印之间没有新数据,那么水印末尾是否会再次发射?

  2. If yes, under the same scenario, will there be another firing at the end of the watermark if accumulatingFiredPanes is changed to discardingFiredPanes?

    如果是,在相同的情况下,如果accumulatingFiredPanes更改为discardingFiredPanes,那么水印末尾是否会再次触发?

1 个解决方案

#1


1  

  1. Yes. There should always be a firing when the watermark passes the end of the window. The early firing panes will be marked as early, and the watermark pane will be marked as on time.

    是。当水印通过窗户的末端时,应始终发射。早期的射击窗格将被标记为早期,水印窗格将被标记为准时。

  2. Yes, currently we always guarantee an on_time pane, which means there will be a firing at the end of the watermark.

    是的,目前我们总是保证一个on_time窗格,这意味着在水印结束时将会触发。

#1


1  

  1. Yes. There should always be a firing when the watermark passes the end of the window. The early firing panes will be marked as early, and the watermark pane will be marked as on time.

    是。当水印通过窗户的末端时,应始终发射。早期的射击窗格将被标记为早期,水印窗格将被标记为准时。

  2. Yes, currently we always guarantee an on_time pane, which means there will be a firing at the end of the watermark.

    是的,目前我们总是保证一个on_time窗格,这意味着在水印结束时将会触发。