什么是更快 - 将pickle字典对象或加载JSON文件加载到字典？

What is faster:

什么是更快：

(A) 'Unpickling' (Loading) a pickled dictionary object, using pickle.load()

（A）'取消'（加载）一个pickle字典对象，使用pickle.load（）

要么

(B) Loading a JSON file to a dictionary using simplejson.load()

（B）使用simplejson.load（）将JSON文件加载到字典

Assuming: The pickled object file exists already in case A, and that the JSON file exists already in case B.

假设：已经在案例A中存在pickle对象文件，并且在案例B中已经存在JSON文件。

1 个解决方案

#1

The speed actually depends on the data, it's content and size.

速度实际上取决于数据，内容和大小。

But, anyway, let's take an example json data and see what is faster (Ubuntu 12.04, python 2.7.3) :

但是，无论如何，让我们以json数据为例，看看速度更快（Ubuntu 12.04，python 2.7.3）：

pickle
泡菜
cPickle
cPickle的
json
JSON
simplejson
simplejson
ujson
ujson
yajl
yajl

Giving this json structure dumped into test.json and test.pickle files:

将此json结构转储到test.json和test.pickle文件中：

{
    "glossary": {
        "title": "example glossary",
        "GlossDiv": {
            "title": "S",
            "GlossList": {
                "GlossEntry": {
                    "ID": "SGML",
                    "SortAs": "SGML",
                    "GlossTerm": "Standard Generalized Markup Language",
                    "Acronym": "SGML",
                    "Abbrev": "ISO 8879:1986",
                    "GlossDef": {
                        "para": "A meta-markup language, used to create markup languages such as DocBook.",
                        "GlossSeeAlso": ["GML", "XML"]
                    },
                    "GlossSee": "markup"
                }
            }
        }
    }
}

Testing script:

测试脚本：

import timeit

import pickle
import cPickle

import json
import simplejson
import ujson
import yajl


def load_pickle(f):
    return pickle.load(f)


def load_cpickle(f):
    return cPickle.load(f)


def load_json(f):
    return json.load(f)


def load_simplejson(f):
    return simplejson.load(f)


def load_ujson(f):
    return ujson.load(f)


def load_yajl(f):
    return yajl.load(f)


print "pickle:"
print timeit.Timer('load_pickle(open("test.pickle"))', 'from __main__ import load_pickle').timeit()

print "cpickle:"
print timeit.Timer('load_cpickle(open("test.pickle"))', 'from __main__ import load_cpickle').timeit()

print "json:"
print timeit.Timer('load_json(open("test.json"))', 'from __main__ import load_json').timeit()

print "simplejson:"
print timeit.Timer('load_simplejson(open("test.json"))', 'from __main__ import load_simplejson').timeit()

print "ujson:"
print timeit.Timer('load_ujson(open("test.json"))', 'from __main__ import load_ujson').timeit()

print "yajl:"
print timeit.Timer('load_yajl(open("test.json"))', 'from __main__ import load_yajl').timeit()

Output:

输出：

pickle:
107.936687946

cpickle:
28.4231381416

json:
31.6450419426

simplejson:
20.5853149891

ujson:
16.9352178574

yajl:
18.9763481617

As you can see, unpickling via pickle is not that fast at all - cPickle is definetely the way to go if you choose pickling/unpickling option. ujson looks promising among these json parsers on this particular data.

正如您所看到的那样，通过pickle进行unpickling并不是那么快 - 如果您选择pickle / unpickling选项，cPickle将是最佳选择。在这些特殊数据中，ujson看起来很有希望在这些json解析器中使用。

Also, json and simplejson libraries load much faster on pypy (see Python JSON Performance).

此外，json和simplejson库在pypy上的加载速度要快得多（参见Python JSON性能）。

#1