我如何知道为运行数据流作业设置了多少个VM?

时间:2023-01-13 09:54:36

I use dataflow service to run my job to read data from GCS and write to BQ tables, jobs run successfully, I'd like to know how or where I can find info how many VMs google set up for my job?

我使用数据流服务来运行我的工作来从GCS读取数据并写入BQ表,成功运行作业,我想知道如何或在哪里可以找到谷歌为我的工作设置了多少个虚拟机?

thank you.

谢谢。

1 个解决方案

#1


5  

When running Cloud Dataflow pipelines using [Blocking]DataflowPipelineRunner, the default number of worker virtual machines is 3.

使用[Blocking] DataflowPipelineRunner运行Cloud Dataflow管道时,默认的工作虚拟机数量为3。

This value can be overridden on the command-line, using --numWorkers parameter.

可以使用--numWorkers参数在命令行上覆盖此值。

All virtual machines, including those created by Cloud Dataflow, are visible in Google Developers Console, under section: Compute, Compute Engine, VM instances.

所有虚拟机(包括由Cloud Dataflow创建的虚拟机)都可在Google Developers Console中的“计算,计算引擎,VM实例”部分中看到。

#1


5  

When running Cloud Dataflow pipelines using [Blocking]DataflowPipelineRunner, the default number of worker virtual machines is 3.

使用[Blocking] DataflowPipelineRunner运行Cloud Dataflow管道时,默认的工作虚拟机数量为3。

This value can be overridden on the command-line, using --numWorkers parameter.

可以使用--numWorkers参数在命令行上覆盖此值。

All virtual machines, including those created by Cloud Dataflow, are visible in Google Developers Console, under section: Compute, Compute Engine, VM instances.

所有虚拟机(包括由Cloud Dataflow创建的虚拟机)都可在Google Developers Console中的“计算,计算引擎,VM实例”部分中看到。