Current we are using Oracle to store logging information. One of the column is a blob that stores an XML payload. We archive the logging schema every week, but we still hit about > 1TB a week.
目前我们使用Oracle来存储日志信息。其中一列是存储XML有效负载的blob。我们每周归档日志模式,但我们仍然每周大约达到1TB。
Goal: Reduce space footprint of database
目标:减少数据库的空间占用
Solution 1: Compress the XML payload before putting it into the BLOB.
解决方案1:在将XML有效负载放入BLOB之前压缩它。
Solution 2: Look for a database (noSQL or other relational DB) that already compresses data on default. JDBC driver must be available.
解决方案2:查找已在默认情况下压缩数据的数据库(noSQL或其他关系数据库)。 JDBC驱动程序必须可用。
Anyone know a database that fits into solution 2? So that we do not need to modify the app, we just need to change the DB and update the JDBC drivers.
有人知道适合解决方案2的数据库吗?因此,我们不需要修改应用程序,我们只需要更改数据库并更新JDBC驱动程序。
3 个解决方案
#1
6
Oracle supports a few different levels of compression - that fits in to solution 2 with minimal effort. (Table / row / lob compression)
Oracle支持几种不同级别的压缩 - 以最小的努力适应解决方案2。 (表/行/高压压缩)
#2
0
All the dbms that I know of that support both compression and JDBC.
我所知道的所有dbms都支持压缩和JDBC。
- Oracle
- SQL Server
- DB2
- Teradata
PostgreSQL relies on the underlying operating system for storage. You could probably build PostgreSQL tablespaces on a compressed filesystem.
PostgreSQL依赖底层操作系统进行存储。您可以在压缩文件系统上构建PostgreSQL表空间。
#3
0
First, I would say if you have enterprise, then partition that xml table and periodically backup and prune it (if partitioned by date range, then remove partitions older than x). If that isn't an option, you may try a noSQL data store.
首先,我会说如果你有企业,那么分区xml表并定期备份和修剪它(如果按日期范围分区,则删除早于x的分区)。如果这不是一个选项,您可以尝试noSQL数据存储。
There are plenty of noSQL options available. One I've played with (not yet production) is mongoDB. It stores the data in a binary JSON format (BSON), and can be compressed. It seems to be making headway with bigger companies as well. I once worked on a similar b-tree flat file system some time ago, was very fast. You won't have the same features as a RDBMS though. Good thing about mongoDB is that its commercially supported by 10gen (again, can't say how good that support is, but that's important to some shops).
有很多noSQL选项可供使用。我玩过(还没有制作)的是mongoDB。它以二进制JSON格式(BSON)存储数据,并且可以进行压缩。它似乎也在与更大的公司取得进展。我曾经在一段时间的同一个b-tree平面文件系统上工作过,速度非常快。但是,您将不具有与RDBMS相同的功能。关于mongoDB的好处是它的商业支持10gen(再次,不能说支持有多好,但这对一些商店来说很重要)。
Here's a good article discussing the pros/cons of the bigger noSQL options out there.
这是一篇很好的文章,讨论那些更大的noSQL选项的优缺点。
And no, I don't work for 10gen ;) Of course do more research and decide what's best for your needs
不,我不为10gen工作;)当然,做更多的研究,并决定什么是最适合您的需求
#1
6
Oracle supports a few different levels of compression - that fits in to solution 2 with minimal effort. (Table / row / lob compression)
Oracle支持几种不同级别的压缩 - 以最小的努力适应解决方案2。 (表/行/高压压缩)
#2
0
All the dbms that I know of that support both compression and JDBC.
我所知道的所有dbms都支持压缩和JDBC。
- Oracle
- SQL Server
- DB2
- Teradata
PostgreSQL relies on the underlying operating system for storage. You could probably build PostgreSQL tablespaces on a compressed filesystem.
PostgreSQL依赖底层操作系统进行存储。您可以在压缩文件系统上构建PostgreSQL表空间。
#3
0
First, I would say if you have enterprise, then partition that xml table and periodically backup and prune it (if partitioned by date range, then remove partitions older than x). If that isn't an option, you may try a noSQL data store.
首先,我会说如果你有企业,那么分区xml表并定期备份和修剪它(如果按日期范围分区,则删除早于x的分区)。如果这不是一个选项,您可以尝试noSQL数据存储。
There are plenty of noSQL options available. One I've played with (not yet production) is mongoDB. It stores the data in a binary JSON format (BSON), and can be compressed. It seems to be making headway with bigger companies as well. I once worked on a similar b-tree flat file system some time ago, was very fast. You won't have the same features as a RDBMS though. Good thing about mongoDB is that its commercially supported by 10gen (again, can't say how good that support is, but that's important to some shops).
有很多noSQL选项可供使用。我玩过(还没有制作)的是mongoDB。它以二进制JSON格式(BSON)存储数据,并且可以进行压缩。它似乎也在与更大的公司取得进展。我曾经在一段时间的同一个b-tree平面文件系统上工作过,速度非常快。但是,您将不具有与RDBMS相同的功能。关于mongoDB的好处是它的商业支持10gen(再次,不能说支持有多好,但这对一些商店来说很重要)。
Here's a good article discussing the pros/cons of the bigger noSQL options out there.
这是一篇很好的文章,讨论那些更大的noSQL选项的优缺点。
And no, I don't work for 10gen ;) Of course do more research and decide what's best for your needs
不,我不为10gen工作;)当然,做更多的研究,并决定什么是最适合您的需求