How do I improve Subversion client update performance? It appears to be disk bound on the client.
如何提高Subversion客户端更新性能?它似乎是客户端上的磁盘绑定。
Details:
- CollabNet Windows client version 1.6.2 (r37639)
- Windows XP SP2
- 3 GB RAM with PF Usage around 1 GB and System Cache of 1.1 GB.
- Disk has write caching enabled
- Update takes 7-15 minutes (when very little to update).
- Checkout has 36,083 directories/files (from svn list)
- Repository has 58,750 revisions.
- Checkout takes about 2.7 GB
- Perf monitor shows % Disk Write time stays near 90% during update.
- Max Disk Read Bytes/sec got up to 12.8M and write got up to 5.2M
- CPU, paging file usage, and network usage are all low.
- Watching the server performance seems to show that it isn't a bottleneck.
CollabNet Windows客户端版本1.6.2(r37639)
3 GB RAM,PF使用率约为1 GB,系统缓存为1.1 GB。
磁盘已启用写入缓存
更新需要7-15分钟(更新时很少)。
Checkout有36,083个目录/文件(来自svn列表)
存储库有58,750个修订版。
结帐大约需要2.7 GB
Perf监视器显示更新期间%磁盘写入时间保持在90%附近。
最大磁盘读取字节数/秒达到12.8M,写入速度达到5.2M
CPU,页面文件使用情况和网络使用率都很低。
观察服务器性能似乎表明它不是瓶颈。
I'm especially interested in answers besides getting a faster disk (especially configuration changes).
除了获得更快的磁盘(特别是配置更改)之外,我对答案特别感兴趣。
Updates from some of the suggestions:
一些建议的更新:
- I need the whole thing so sparse directories won't work.
- Another client (TortoiseSVN) takes 7 minutes also
- TortoiseSVN icon overlays have be configured so they don't cause the problem.
- Anti-virus is configured to to skip that directory is it isn't causing the problem.
我需要整个事情,所以稀疏的目录将无法正常工作。
另一个客户(TortoiseSVN)也需要7分钟
TortoiseSVN图标叠加层已配置,因此它们不会导致问题。
防病毒配置为跳过该目录,因为它不会导致问题。
8 个解决方案
#1
I experience exatly the same thing. Recently replaced Perforce with svn, but if we cannot overcome the performance problems on Windows me must consider another tool. Using svn 1.6.6, Win XP and Vista clients. RedHat server. My observations matches yours:
我经历了同样的事情。最近用svn取代了Perforce,但如果我们无法克服Windows上的性能问题,我必须考虑另一种工具。使用svn 1.6.6,Win XP和Vista客户端。 RedHat服务器。我的观察与你的观点相符:
- Huge disk-write activity.
- Antivirus not a bottleneck.
- No matter witch svn-clients are used.
- No server or network bottleneck.
巨大的磁盘写入活动。
防病毒不是瓶颈。
无论使用svn-clients是什么。
没有服务器或网络瓶颈。
Complementary info More than 3 times faster operations on:
补充信息操作速度提高3倍以上:
- Linux (Ubuntu).
- Linux (Ubuntu) running on VirtualBox at Win Vista host.
- Win XP running on VMWare at RedHat host.
在Win Vista主机上运行VirtualBox的Linux(Ubuntu)。
在RedHat主机上的VMWare上运行Win XP。
#2
Do you need every bit of the repository on your working copy? If you truly only care about particular portions of the tree, look into Subversion's Sparse Directories (a.k.a. "Sparse Checkouts") feature. It allows you to manipulate your working copy so it only contains those directories of interest.
您是否需要工作副本上的每个存储库?如果您真的只关心树的特定部分,请查看Subversion的稀疏目录(a.k.a.“稀疏检查”)功能。它允许您操作您的工作副本,因此它只包含那些感兴趣的目录。
Just as an example, you might use this to prune documentation, installer-related files, etc. Depending on what you truly need on your local machine, embracing this approach could make a serious dent in your wait times.
举个例子,您可以使用它来修剪文档,与安装程序相关的文件等。根据您在本地计算机上真正需要的内容,采用这种方法可能会严重影响您的等待时间。
#3
Try svn client version 1.5.. It helped me on my Vista laptop. Versions 1.6. are extremely slow.
尝试svn客户端版本1.5 ..它帮助我在我的Vista笔记本电脑上。版本1.6。非常慢。
#4
This is more likely to be your network and the amount of data moved as well as your client. Are you using Tortoise? I find it to be a bit slow myself when moving that much data!
这更有可能是您的网络和移动的数据量以及您的客户端。你在用Tortoise吗?移动那么多数据时,我发现自己有点慢!
#5
Are you using TortoiseSVN? If so, the Icon Overlays do slow down operations. If you go to TortoiseSVN Settings/Icon Overlays there are several settings you can tweak to control the level to which you want to use the Overlays, including turning them off completely. See if that affects your performance.
你在用TortoiseSVN吗?如果是这样,图标叠加确实会降低操作速度。如果你去TortoiseSVN设置/图标叠加,你可以调整几个设置来控制你想要使用叠加层的级别,包括完全关闭它们。看看这是否会影响您的表现。
#6
Do you run a virus checker that uses on-access scanning? That can really make it crawl. If so, turn it off and see if that helps. Most scanners will have a way to exclude specific directories if that helps.
您是否运行使用按访问扫描的病毒检查程序?这真的可以让它爬行。如果是这样,请将其关闭,看看是否有帮助。如果有帮助,大多数扫描仪都可以排除特定目录。
#7
Nobody seems to be pointing out the one reason that I often consider a design flaw. Subversion creates a second "pristine" copy of the checkout for offline operations. If you're checking out 4G of files, it's actually writing 8G to disk.
似乎没有人指出我经常考虑设计缺陷的一个原因。 Subversion为离线操作创建了结帐的第二个“原始”副本。如果您正在检查4G文件,它实际上是将8G写入磁盘。
Compare a checkout to an export. That will show you the massive difference when writing those second copies.
将结帐与导出进行比较。这将显示编写第二份副本时的巨大差异。
There's nothing you can do about that.
你无能为力。
#8
Upgrade to svn 1.7
升级到svn 1.7
From Discussion of Slow Performance of SVN Update:
从SVN更新缓慢性能的讨论:
The update process in svn 1.6 goes something like this:
svn 1.6中的更新过程如下:
- search the entire working copy, to see what's there at the moment, and locking it so no one changes the answer during the next steps
搜索整个工作副本,查看当前有什么,并锁定它,以便在接下来的步骤中没有人更改答案
- tell that to the server
告诉服务器
- receive from the server whatever new stuff you need, applying the changes to the files as you go
从服务器接收您需要的任何新内容,随时将更改应用于文件
- recurse over the entire working copy again, unlocking it
再次递归整个工作副本,解锁它
If there are many directories and files, steps 1 and 4 can take up a lot of time. This would be consistent with your observation of long delays with no network traffic.
如果有许多目录和文件,步骤1和4可能会占用大量时间。这与您在没有网络流量的情况下长时间延迟的观察结果一致。
Working copy format was changed in svn 1.7. Now all meta information is stored in SQLite database in root folder of working copy and there is no need to perform steps 1 and 4 any more which consumed most of the time durring svn update.
工作副本格式在svn 1.7中已更改。现在所有元信息都存储在工作副本的根文件夹中的SQLite数据库中,并且不再需要执行步骤1和4,这些步骤在svn更新期间消耗了大部分时间。
#1
I experience exatly the same thing. Recently replaced Perforce with svn, but if we cannot overcome the performance problems on Windows me must consider another tool. Using svn 1.6.6, Win XP and Vista clients. RedHat server. My observations matches yours:
我经历了同样的事情。最近用svn取代了Perforce,但如果我们无法克服Windows上的性能问题,我必须考虑另一种工具。使用svn 1.6.6,Win XP和Vista客户端。 RedHat服务器。我的观察与你的观点相符:
- Huge disk-write activity.
- Antivirus not a bottleneck.
- No matter witch svn-clients are used.
- No server or network bottleneck.
巨大的磁盘写入活动。
防病毒不是瓶颈。
无论使用svn-clients是什么。
没有服务器或网络瓶颈。
Complementary info More than 3 times faster operations on:
补充信息操作速度提高3倍以上:
- Linux (Ubuntu).
- Linux (Ubuntu) running on VirtualBox at Win Vista host.
- Win XP running on VMWare at RedHat host.
在Win Vista主机上运行VirtualBox的Linux(Ubuntu)。
在RedHat主机上的VMWare上运行Win XP。
#2
Do you need every bit of the repository on your working copy? If you truly only care about particular portions of the tree, look into Subversion's Sparse Directories (a.k.a. "Sparse Checkouts") feature. It allows you to manipulate your working copy so it only contains those directories of interest.
您是否需要工作副本上的每个存储库?如果您真的只关心树的特定部分,请查看Subversion的稀疏目录(a.k.a.“稀疏检查”)功能。它允许您操作您的工作副本,因此它只包含那些感兴趣的目录。
Just as an example, you might use this to prune documentation, installer-related files, etc. Depending on what you truly need on your local machine, embracing this approach could make a serious dent in your wait times.
举个例子,您可以使用它来修剪文档,与安装程序相关的文件等。根据您在本地计算机上真正需要的内容,采用这种方法可能会严重影响您的等待时间。
#3
Try svn client version 1.5.. It helped me on my Vista laptop. Versions 1.6. are extremely slow.
尝试svn客户端版本1.5 ..它帮助我在我的Vista笔记本电脑上。版本1.6。非常慢。
#4
This is more likely to be your network and the amount of data moved as well as your client. Are you using Tortoise? I find it to be a bit slow myself when moving that much data!
这更有可能是您的网络和移动的数据量以及您的客户端。你在用Tortoise吗?移动那么多数据时,我发现自己有点慢!
#5
Are you using TortoiseSVN? If so, the Icon Overlays do slow down operations. If you go to TortoiseSVN Settings/Icon Overlays there are several settings you can tweak to control the level to which you want to use the Overlays, including turning them off completely. See if that affects your performance.
你在用TortoiseSVN吗?如果是这样,图标叠加确实会降低操作速度。如果你去TortoiseSVN设置/图标叠加,你可以调整几个设置来控制你想要使用叠加层的级别,包括完全关闭它们。看看这是否会影响您的表现。
#6
Do you run a virus checker that uses on-access scanning? That can really make it crawl. If so, turn it off and see if that helps. Most scanners will have a way to exclude specific directories if that helps.
您是否运行使用按访问扫描的病毒检查程序?这真的可以让它爬行。如果是这样,请将其关闭,看看是否有帮助。如果有帮助,大多数扫描仪都可以排除特定目录。
#7
Nobody seems to be pointing out the one reason that I often consider a design flaw. Subversion creates a second "pristine" copy of the checkout for offline operations. If you're checking out 4G of files, it's actually writing 8G to disk.
似乎没有人指出我经常考虑设计缺陷的一个原因。 Subversion为离线操作创建了结帐的第二个“原始”副本。如果您正在检查4G文件,它实际上是将8G写入磁盘。
Compare a checkout to an export. That will show you the massive difference when writing those second copies.
将结帐与导出进行比较。这将显示编写第二份副本时的巨大差异。
There's nothing you can do about that.
你无能为力。
#8
Upgrade to svn 1.7
升级到svn 1.7
From Discussion of Slow Performance of SVN Update:
从SVN更新缓慢性能的讨论:
The update process in svn 1.6 goes something like this:
svn 1.6中的更新过程如下:
- search the entire working copy, to see what's there at the moment, and locking it so no one changes the answer during the next steps
搜索整个工作副本,查看当前有什么,并锁定它,以便在接下来的步骤中没有人更改答案
- tell that to the server
告诉服务器
- receive from the server whatever new stuff you need, applying the changes to the files as you go
从服务器接收您需要的任何新内容,随时将更改应用于文件
- recurse over the entire working copy again, unlocking it
再次递归整个工作副本,解锁它
If there are many directories and files, steps 1 and 4 can take up a lot of time. This would be consistent with your observation of long delays with no network traffic.
如果有许多目录和文件,步骤1和4可能会占用大量时间。这与您在没有网络流量的情况下长时间延迟的观察结果一致。
Working copy format was changed in svn 1.7. Now all meta information is stored in SQLite database in root folder of working copy and there is no need to perform steps 1 and 4 any more which consumed most of the time durring svn update.
工作副本格式在svn 1.7中已更改。现在所有元信息都存储在工作副本的根文件夹中的SQLite数据库中,并且不再需要执行步骤1和4,这些步骤在svn更新期间消耗了大部分时间。