如果bash脚本中不存在原子创建文件

时间:2022-09-16 23:45:44

In system call open(), if I open with O_CREAT | O_EXCL, the system call ensures that the file will only be created if it does not exist. The atomicity is guaranteed by the system call. Is there a similar way to create a file in an atomic fashion from a bash script?

在系统调用open()中,如果我用O_CREAT打开| O_EXCL,系统调用确保仅在文件不存在时才创建该文件。系统调用保证了原子性。是否有类似的方法从bash脚本以原子方式创建文件?

UPDATE: I found two different atomic ways

更新:我发现了两种不同的原子方式

  1. Use set -o noclobber. Then you can use > operator atomically.
  2. 使用set -o noclobber。然后你可以原子地使用> operator。
  3. Just use mkdir. Mkdir is atomic
  4. 只需使用mkdir。 Mkdir是原子的

5 个解决方案

#1


18  

A 100% pure bash solution:

100%纯粹的bash解决方案:

set -o noclobber
{ > file ; } &> /dev/null

This command creates a file named file if there's no existent file named file. If there's a file named file, then do nothing (but return a non-zero return code).

如果没有名为file的现有文件,则此命令将创建名为file的文件。如果有一个名为file的文件,则不执行任何操作(但返回非零返回码)。

Pros wrt the touch command:

优点与触摸命令:

  • Doesn't update timestamp if file already existed
  • 如果文件已存在,则不更新时间戳
  • 100% bash builtin
  • 内置100%bash
  • Return code as expected: fail if file already existed or if file couldn't be created; success if file didn't exist and was created.
  • 按预期返回代码:如果文件已存在或无法创建文件,则失败;如果文件不存在并且已创建,则成功。

Cons:

缺点:

  • need to set the noclobber option (but it's okay in a script, if you're careful with redirections, or unset it afterwards).
  • 需要设置noclobber选项(但是如果你小心重定向,或者之后取消设置,那么它在脚本中是可以的)。

I guess this solution is really the bash counterpart of the open system call with O_CREAT | O_EXCL.

我想这个解决方案实际上是使用O_CREAT |打开系统调用的bash对应物O_EXCL。

#2


4  

Here's a bash function using the mv -n trick:

这是使用mv -n技巧的bash函数:

function mkatomic() {
  f="$(mktemp)"
  mv -n "$f" "$1"
  if [ -e "$f" ]; then
    rm "$f"
    echo "ERROR: file exists:" "$1" >&2
    return 1
  fi
}

Examples:

例子:

$ mkatomic foo
$ wc -c foo
0 foo
$ mkatomic foo
ERROR: file exists: foo

#3


2  

Just to be clear, ensuring the file will only be created if it doesn't exist is not the same thing as atomicity. The operation is atomic if and only if, when two or more separate threads attempt to do the same thing at the same time, exactly one will succeed and all others will fail.

需要明确的是,确保仅在不存在的情况下创建文件与原子性不同。当且仅当两个或多个单独的线程同时尝试执行相同的操作时,该操作才是原子操作,一个将成功,所有其他线程将失败。

The best way I know of to create a file atomically in a shell script follows this pattern (and it's not perfect):

我知道在shell脚本中以原子方式创建文件的最佳方式遵循此模式(并且它并不完美):

  1. create a file that has an extremely high chance of not existing (using a decent random number selection or something in the file name), and place some unique content in it (something that no other thread would have - again, a random number or something)
  2. 创建一个很有可能不存在的文件(使用一个像样的随机数选择或文件名中的东西),并在其中放置一些独特的内容(没有其他线程可以拥有的东西 - 再次,一个随机数或其他东西)
  3. verify that the file exists and contains the contents you expect it to
  4. 验证文件是否存在并包含您期望的内容
  5. create a hard link from that file to the desired file
  6. 创建从该文件到所需文件的硬链接
  7. verify that the desired file contains the expected contents
  8. 验证所需文件是否包含预期内容

In particular, touch is not atomic, since it will create the file if it's not there, or simply update the timestamp. You might be able to play games with different timestamps, but reading and parsing a timestamp to see if you "won" the race is harder than the above. mkdir can be atomic, but you would have to check the return code, because otherwise, you can only tell that "yes, the directory was created, but I don't know which thread won". If you're on a file system that doesn't support hard links, you might have to settle for a less ideal solution.

特别是,触摸不是原子的,因为它会创建文件(如果它不存在),或者只是更新时间戳。您可能能够玩具有不同时间戳的游戏,但是阅读和解析时间戳以查看您是否“赢得”比赛比上述更难。 mkdir可以是原子的,但你必须检查返回代码,否则,你只能告诉“是的,目录已创建,但我不知道哪个线程赢了”。如果您使用的文件系统不支持硬链接,则可能需要采用不太理想的解决方案。

#4


2  

You could create it under a randomly-generated name, then rename (mv -n random desired) it into place with the desired name. The rename will fail if the file already exists.

您可以在随机生成的名称下创建它,然后使用所需名称将其重命名(mv -n random desired)到位。如果文件已存在,则重命名将失败。

Like this:

喜欢这个:

#!/bin/bash

touch randomFileName
mv -n randomFileName lockFile

if [ -e randomFileName ] ; then
    echo "Failed to acquired lock"
else
    echo "Acquired lock"
fi

#5


-2  

touch is the command you are looking for. It updates timestamps of the provided file if the file exists or creates it if it doesn't.

触摸是您正在寻找的命令。如果文件存在,它会更新所提供文件的时间戳,如果不存在则会创建它。

#1


18  

A 100% pure bash solution:

100%纯粹的bash解决方案:

set -o noclobber
{ > file ; } &> /dev/null

This command creates a file named file if there's no existent file named file. If there's a file named file, then do nothing (but return a non-zero return code).

如果没有名为file的现有文件,则此命令将创建名为file的文件。如果有一个名为file的文件,则不执行任何操作(但返回非零返回码)。

Pros wrt the touch command:

优点与触摸命令:

  • Doesn't update timestamp if file already existed
  • 如果文件已存在,则不更新时间戳
  • 100% bash builtin
  • 内置100%bash
  • Return code as expected: fail if file already existed or if file couldn't be created; success if file didn't exist and was created.
  • 按预期返回代码:如果文件已存在或无法创建文件,则失败;如果文件不存在并且已创建,则成功。

Cons:

缺点:

  • need to set the noclobber option (but it's okay in a script, if you're careful with redirections, or unset it afterwards).
  • 需要设置noclobber选项(但是如果你小心重定向,或者之后取消设置,那么它在脚本中是可以的)。

I guess this solution is really the bash counterpart of the open system call with O_CREAT | O_EXCL.

我想这个解决方案实际上是使用O_CREAT |打开系统调用的bash对应物O_EXCL。

#2


4  

Here's a bash function using the mv -n trick:

这是使用mv -n技巧的bash函数:

function mkatomic() {
  f="$(mktemp)"
  mv -n "$f" "$1"
  if [ -e "$f" ]; then
    rm "$f"
    echo "ERROR: file exists:" "$1" >&2
    return 1
  fi
}

Examples:

例子:

$ mkatomic foo
$ wc -c foo
0 foo
$ mkatomic foo
ERROR: file exists: foo

#3


2  

Just to be clear, ensuring the file will only be created if it doesn't exist is not the same thing as atomicity. The operation is atomic if and only if, when two or more separate threads attempt to do the same thing at the same time, exactly one will succeed and all others will fail.

需要明确的是,确保仅在不存在的情况下创建文件与原子性不同。当且仅当两个或多个单独的线程同时尝试执行相同的操作时,该操作才是原子操作,一个将成功,所有其他线程将失败。

The best way I know of to create a file atomically in a shell script follows this pattern (and it's not perfect):

我知道在shell脚本中以原子方式创建文件的最佳方式遵循此模式(并且它并不完美):

  1. create a file that has an extremely high chance of not existing (using a decent random number selection or something in the file name), and place some unique content in it (something that no other thread would have - again, a random number or something)
  2. 创建一个很有可能不存在的文件(使用一个像样的随机数选择或文件名中的东西),并在其中放置一些独特的内容(没有其他线程可以拥有的东西 - 再次,一个随机数或其他东西)
  3. verify that the file exists and contains the contents you expect it to
  4. 验证文件是否存在并包含您期望的内容
  5. create a hard link from that file to the desired file
  6. 创建从该文件到所需文件的硬链接
  7. verify that the desired file contains the expected contents
  8. 验证所需文件是否包含预期内容

In particular, touch is not atomic, since it will create the file if it's not there, or simply update the timestamp. You might be able to play games with different timestamps, but reading and parsing a timestamp to see if you "won" the race is harder than the above. mkdir can be atomic, but you would have to check the return code, because otherwise, you can only tell that "yes, the directory was created, but I don't know which thread won". If you're on a file system that doesn't support hard links, you might have to settle for a less ideal solution.

特别是,触摸不是原子的,因为它会创建文件(如果它不存在),或者只是更新时间戳。您可能能够玩具有不同时间戳的游戏,但是阅读和解析时间戳以查看您是否“赢得”比赛比上述更难。 mkdir可以是原子的,但你必须检查返回代码,否则,你只能告诉“是的,目录已创建,但我不知道哪个线程赢了”。如果您使用的文件系统不支持硬链接,则可能需要采用不太理想的解决方案。

#4


2  

You could create it under a randomly-generated name, then rename (mv -n random desired) it into place with the desired name. The rename will fail if the file already exists.

您可以在随机生成的名称下创建它,然后使用所需名称将其重命名(mv -n random desired)到位。如果文件已存在,则重命名将失败。

Like this:

喜欢这个:

#!/bin/bash

touch randomFileName
mv -n randomFileName lockFile

if [ -e randomFileName ] ; then
    echo "Failed to acquired lock"
else
    echo "Acquired lock"
fi

#5


-2  

touch is the command you are looking for. It updates timestamps of the provided file if the file exists or creates it if it doesn't.

触摸是您正在寻找的命令。如果文件存在,它会更新所提供文件的时间戳,如果不存在则会创建它。