如何使用java从Json文件导入Mongodb数据

I am struggling with importing data into Mongodb from a Json file.
I can do the same in command line by using mongoimport command.
I explored and tried lot but not able to import from Json file using java.

我正在努力将数据从Json文件导入Mongodb。我可以使用mongoimport命令在命令行中执行相同的操作。我探索并尝试了很多,但无法使用java从Json文件导入。

sample.json

    { "test_id" : 1245362, "name" : "ganesh", "age" : "28", "Job" : 
       {"company name" : "company1", "designation" : "SSE" } 
    }

    { "test_id" : 254152, "name" : "Alex", "age" : "26", "Job" :
       {"company name" : "company2", "designation" : "ML" } 
    }

Thank for your time. ~Ganesh~

谢谢你的时间。 〜Ganesh神〜

6 个解决方案

#1

Suppose you can read the JSON string respectively. For example, you read the first JSON text

假设您可以分别读取JSON字符串。例如,您阅读了第一个JSON文本

{ "test_id" : 1245362, "name" : "ganesh", "age" : "28", "Job" : 
   {"company name" : "company1", "designation" : "SSE" } 
}

and assign it to a variable (String json1), the next step is to parse it,

并将其分配给变量(String json1),下一步是解析它,

DBObject dbo = (DBObject) com.mongodb.util.JSON.parse(json1);

put all dbo into a list,

将所有dbo放入列表中

List<DBObject> list = new ArrayList<>();
list.add(dbo);

then save them into database:

然后将它们保存到数据库:

new MongoClient().getDB("test").getCollection("collection").insert(list);

EDIT:

In the newest MongoDB Version you have to use Documents instead of DBObject, and the methods for adding the object look different now. Here's an updated example:

在最新的MongoDB版本中,您必须使用Documents而不是DBObject,现在添加对象的方法看起来不同。这是一个更新的例子:

Imports are:

import com.mongodb.MongoClient;
import com.mongodb.client.MongoDatabase;
import org.bson.Document;

The code would like this (refering to the text above the EDIT):

代码是这样的(参考编辑上面的文字):

Document doc = Document.parse(json1);
new MongoClient().getDataBase("db").getCollection("collection").insertOne(doc);

you can also do it the way with the list. but then you need

您也可以按照列表的方式进行操作。但是你需要

new MongoClient().getDataBase("db").getCollection("collection").insertMany(list);

But I think there is a problem with this solution. When you type:

但我认为这个解决方案存在问题。键入时:

db.collection.find()

in the mongo shell to get all objects in the collection, the result looks like the following:

在mongo shell中获取集合中的所有对象,结果如下所示:

{ "_id" : ObjectId("56a0d2ddbc7c512984be5d97"),
    "test_id" : 1245362, "name" : "ganesh", "age" : "28", "Job" :
        { "company name" : "company1", "designation" : "SSE" 
    }
}

which is not exactly the same as before.

这和以前不完全一样。

#2

Runtime r = Runtime.getRuntime();

运行时r = Runtime.getRuntime();

Process p = null;

过程p = null;

--dir is the path to where your mongoimport is.

--dir是您的mongoimport所在的路径。

File dir=new File("C:\Program Files\MongoDB\Server\3.2\bin");

File dir = new File(“C:\ Program Files \ MongoDB \ Server \ 3.2 \ bin”);

--this line will open your shell in giving dir, the command for import is exactly same as you use mongoimport in command promote

- 这行将在给出dir时打开你的shell,导入的命令与你在命令提升中使用mongoimport完全一样

p = r.exec("c:\windows\system32\cmd.exe /c mongoimport --db mydb --collection student --type csv --file student.csv --headerline" ,null,dir);

p = r.exec(“c:\ windows \ system32 \ cmd.exe / c mongoimport --db mydb --collection student --type csv --file student.csv --headerline”,null,dir);

#3

Had a similar "problem" myself and ended up using Jackson with POJO databinding, and Morphia.

我自己也遇到了类似的“问题”,并最终使用了POJO数据绑定和Morphia。

While this sound a bit like cracking a nut with a sledgehammer, it is actually very easy to use, robust and quite performant and easy to maintain code wise.

虽然这听起来有点像用大锤敲打螺母,但它实际上非常容易使用,坚固且性能相当且易于维护代码。

Small caveat: You need to map your test_id field to MongoDB's _id if you want to reuse it.

小警告:如果要重用它,则需要将test_id字段映射到MongoDB的_id。

Step 1: Create an annotated bean

You need to hint Jackson how to map the data from a JSON file to a POJO. I shortened the class a bit for the sake of readability:

您需要提示Jackson如何将数据从JSON文件映射到POJO。为了便于阅读,我将课程缩短了一点:

@JsonRootName(value="person")
@Entity
public class Person {

  @JsonProperty(value="test_id")
  @Id
  Integer id;

  String name;

  public Integer getId() {
    return id;
  }

  public void setId(Integer id) {
    this.id = id;
  }

  public String getName() {
    return name;
  }

  public void setName(String name) {
    this.name = name;
  }

}

As for the embedded document Job, please have a look at the POJO data binding examples linked.

至于嵌入式文档Job,请查看链接的POJO数据绑定示例。

Step 2: Map the POJO and create a datastore

Somewhere during your application initialization, you need to map the annotated POJO. Since you already should have a MongoClient, I am going to reuse that ;)

在应用程序初始化期间的某个地方,您需要映射带注释的POJO。既然你已经有了MongoClient,我将重用它;)

Morphia morphia = new Morphia();
morphia.map(Person.class);

/* You can reuse this datastore */
Datastore datastore = morphia.createDatastore(mongoClient, "myDatabase");

/* 
 * Jackson's ObjectMapper, which is reusable, too,
 * does all the magic.
 */
ObjectMapper mapper = new ObjectMapper();

Do the actual importing

Now importing a given JSON file becomes as easy as

现在导入给定的JSON文件变得如此简单

public Boolean importJson(Datastore ds, ObjectMapper mapper, String filename) {

    try {           
        JsonParser parser = new JsonFactory().createParser(new FileReader(filename));
        Iterator<Person> it = mapper.readValues(parser, Person.class);

        while(it.hasNext()) {
            ds.save(it.next());
        }

        return Boolean.TRUE;

    } catch (JsonParseException e) {
        /* Json was invalid, deal with it here */
    } catch (JsonMappingException e) {
        /* Jackson was not able to map
         * the JSON values to the bean properties,
         * possibly because of
         * insufficient mapping information.
         */
    } catch (IOException e) {
        /* Most likely, the file was not readable
         * Should be rather thrown, but was
         * cought for the sake of showing what can happen
         */
    }

    return Boolean.FALSE;
}

With a bit of refatcoring, this can be converted in a generic importer for Jackson annotated beans. Obviously, I left out some special cases, but this would out of the scope of this answer.

通过一些refatcoring,可以在Jackson注释bean的通用导入器中进行转换。显然,我遗漏了一些特殊情况,但这超出了这个答案的范围。

#4

With 3.2 driver, if you have a mongo collection and a collection of json documents e.g:

使用3.2驱动程序,如果你有一个mongo集合和一组json文档,例如:

MongoCollection<Document> collection = ...
List<String> jsons = ...

You can insert individually:

您可以单独插入:

jsons.stream().map(Document::parse).forEach(collection::insertOne);

or bulk:

collection.insertMany(
        jsons.stream().map(Document::parse).collect(Collectors.toList())
); 

#5

I just faced this issue today and solved it in another different way while none here satisfied me, so enjoy my extra contribution. Performances are sufficient to export 30k documents and import them in my Springboot app for integration test cases (takes a few seconds).

我今天刚刚面对这个问题并以另一种方式解决了这个问题,而没有人满意我,所以尽情享受我的额外贡献。性能足以导出30k文档并将其导入我的Springboot应用程序以进行集成测试用例(需要几秒钟)。

First, the way your export your data in the first place matters. I wanted a file where each line contains 1 document that I can parse in my java app.

首先,您首先导出数据的方式很重要。我想要一个文件,其中每行包含1个文档,我可以在我的Java应用程序中解析。

mongo db --eval 'db.data.find({}).limit(30000).forEach(function(f){print(tojson(f, "", true))})' --quiet > dataset.json

Then I get the file from my resources folder, parse it, extract lines, and process them with mongoTemplate. Could use a buffer.

然后我从我的资源文件夹中获取文件,解析它,提取行,并使用mongoTemplate处理它们。可以使用缓冲区。

@Autowired    
private MongoTemplate mongoTemplate;

public void createDataSet(){
    mongoTemplate.dropCollection("data");
    try {
        InputStream inputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(DATASET_JSON);
        List<Document> documents = new ArrayList<>();
        String line;
        InputStreamReader isr = new InputStreamReader(inputStream, Charset.forName("UTF-8"));
        BufferedReader br = new BufferedReader(isr);
        while ((line = br.readLine()) != null) {
            documents.add(Document.parse(line));
        }
        mongoTemplate.insert(documents,"data");


    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

#6

List<Document> jsonList = new ArrayList<Document>();
net.sf.json.JSONArray array = net.sf.json.JSONArray.fromObject(json);
for (Object object : array) {
    net.sf.json.JSONObject jsonStr = (net.sf.json.JSONObject)JSONSerializer.toJSON(object);
    Document jsnObject = Document.parse(jsonStr.toString()); 
    jsonList.add(jsnObject);
}
collection.insertMany(jsonList);