Titan-红号楼宗谱案例

时间：2023-03-09 07:03:45
Titan-红号楼宗谱案例

一. 简介

  1. titan：存储，查询图形结构的数据库。分布式集群环境下，可支持数以千亿级别的点和边，同时支持上千个并发的实时的复杂图形遍历，支持ACID事务。
  2. 架构：支持以下3方面的*组合

    （1）节点和边的存储：

         Apache Cassandra

         Apache HBase

         Oracle BerkeleyDB（测试使用）

    （2）图形分析组件：

         Spark

         Giraph

         Hadoop

    （3）地理，数值，全文检索支持

         ElasticSearch

         Solr

         Lucene

二. titan-hbase-es部署范例

  1. 版本对应

    （1）titan:0.54-hadoop2

    （2）hbase:1.1.5 (版本相对宽松)

    （3）elasticsearch:1.4

    （4）rexster-server:2.6

  2. titan server部署

    （1）rexster-server解压

    （2）修改config/rexster.xml文件,添加如下内容

    <graph>
    
    <graph-enabled>true</graph-enabled>
    
    <graph-name>titanexample</graph-name>
    
    <graph-type>com.thinkaurelius.titan.tinkerpop.rexster.TitanGraphConfiguration</graph-type>
    
    <graph-location></graph-location>
    
    <graph-read-only>false</graph-read-only>
    
    <properties>
    
	<storage.backend>hbase</storage.backend>
    
	<storage.hostname>localhost:2181,localhost:2182,localhost:2183</storage.hostname>
    
	<storage.hbase.table>facebook</storage.hbase.table>
    
	<index.search.backend>elasticsearch</index.search.backend>
    
	<index.search.elasticsearch.client-only>true</index.search.elasticsearch.client-only>
    
	<index.search.hostname>127.0.0.1</index.search.hostname>
    
	<index.search.index-name>facebook</index.search.index-name>
    
    </properties>
    
    <extensions>
    
    <allows>
    
	<allow>tp:gremlin</allow>
    
    </allows>
    
    </extensions>
    
</graph>

（3）复制titan jar包到rexster lib

```shell

cp   TITAN_HOME/lib/*    REXSTER_HOME/ext/titan/

```

（4）删除rexster/lib下的lucene相关jar包，会与titan的引起冲突

（5）开启rexster

```shell

./bin/rexster.sh -s -c ./config/rexster.xml

```

（6）访问http://ip:8182/graphs/titanexample，REST

三. titan server接口

  1. RexPro二进制协议

    public class TestClient {
    
    public static void main(String[] args) throws Exception {
    
        RexsterClient client = RexsterClientFactory.open("localhost", "titanexample");
    
        List<Map<String,Object>> result;
    
        result = client.execute("aa=g.V.has('name','saturn');aa.next()");
    
        //result = client.execute("g.getManagementSystem().get(‘cache.db-cache’)");
    
        // result.toString(): [{name="jupiter"}]
    
        System.out.println(result);
    
        client.close();
    
    }
    
}
    <dependency>
    
    <groupId>com.tinkerpop.rexster</groupId>
    
    <artifactId>rexster-protocol</artifactId>
    
    <version>2.6.0</version>
    
</dependency>

  2. thinkerpop - REST协议

四. “红楼梦宗谱”示例

  1. 编写blueprint脚本，设置schema，索引，添加将节点数据

    com.thinkaurelius.titan.core.schema.TitanManagement mgmt = g.getManagementSystem();
    
//点的属性名
    
com.thinkaurelius.titan.core.PropertyKey name = mgmt.makePropertyKey("name").dataType(String.class).make();
    
com.thinkaurelius.titan.core.PropertyKey age = mgmt.makePropertyKey("age").dataType(Integer.class).make();
    
// 点的标签名
    
mgmt.makeVertexLabel("people").make();  //该点表示人
    
mgmt.makeVertexLabel("hobby").make();   //该点是一个运动
    
// 给点的姓名年龄建索引
    
mgmt.buildIndex("name",Vertex.class).addKey(name).unique().buildCompositeIndex(); // "search"是配置文件中的标识符
    
mgmt.buildIndex("vertices",Vertex.class).addKey(age).buildMixedIndex("search");   // mixedIndex是外部索引，用es存储索引

// 边的属性
    
mgmt.makeEdgeLabel("father").multiplicity(com.thinkaurelius.titan.core.Multiplicity.MANY2ONE).make();
    
mgmt.makeEdgeLabel("mother").multiplicity(com.thinkaurelius.titan.core.Multiplicity.MANY2ONE).make();
    
mgmt.makeEdgeLabel("hobby").multiplicity(com.thinkaurelius.titan.core.Multiplicity.MULTI).make();
    
com.thinkaurelius.titan.core.PropertyKey time = mgmt.makePropertyKey("time").dataType(Integer.class).make();
    
com.thinkaurelius.titan.core.EdgeLabel love = mgmt.makeEdgeLabel("love").signature(time).make();//什么时候确立爱情挂席
    
mgmt.buildEdgeIndex(love,"lovetime", Direction.BOTH, com.thinkaurelius.titan.core.Order.DESC,time);
    
mgmt.commit();

//插入点
    
com.thinkaurelius.titan.core.TitanTransaction tx = g.newTransaction();
    
Vertex jiazheng = tx.addVertexWithLabel("people");   // 贾政
    
jiazheng.setProperty("name","贾政");
    
jiazheng.setProperty("age",48);
    
Vertex jiabaoyu = tx.addVertexWithLabel("people");   // 贾宝玉
    
jiabaoyu.setProperty("name","贾宝玉");
    
jiabaoyu.setProperty("age",18);
    
Vertex wangfuren = tx.addVertexWithLabel("people");  // 王夫人
    
wangfuren.setProperty("name","王夫人");
    
wangfuren.setProperty("age",47);
    
Vertex xuebaochai = tx.addVertexWithLabel("people");  // 薛宝钗
    
xuebaochai.setProperty("name","薛宝钗");
    
xuebaochai.setProperty("age",17);

Vertex cixiu = tx.addVertexWithLabel("hobby");
    
cixiu.setProperty("name","刺绣");
    
Vertex zuoshi = tx.addVertexWithLabel("hobby");
    
zuoshi.setProperty("name","作诗");

//插入边
    
jiabaoyu.addEdge("father",jiazheng);
    
jiabaoyu.addEdge("mother",wangfuren);
    
ElementHelper.setProperties(jiabaoyu.addEdge("love",xuebaochai),"time",1001);   // 贾宝玉爱林黛玉，"time"属性为1001
    
wangfuren.addEdge("hobby",cixiu);
    
xuebaochai.addEdge("hobby",zuoshi);

tx.commit();

  2. 通过RexPro协议发送脚本到服务器

    public static void main(String[] args) throws Exception {
    
    RexsterClient client = RexsterClientFactory.open("localhost", "titanexample");
    
    List<Map<String,Object>> result;
    
    BufferedReader br = new BufferedReader(new InputStreamReader(Thread.currentThread().getContextClassLoader().getResourceAsStream("facebook-gremlin")));
    
    String str="";
    
    StringBuffer sb = new StringBuffer();
    
    while((str = br.readLine())!=null){
    
        sb.append(str);
    
    }
    
    System.out.println(sb.toString());
    
    result = client.execute(sb.toString());
    
    System.out.println(result);
    
    client.close();
    
}

  3. rest api

    curl -XGET http://localhost:8182/graphs/titanexample/vertices   #查看所有边
    
curl -XGET http://localhost:8182/graphs/titanexample/edges
    
curl -XGET http://localhost:8182/graphs/titanexample/keyindices
    
curl -XGET http://localhost:8182/graphs/titanexample/vertices/16400/in  #查询节点的入射边
    
curl -XPOST http://localhost:8182/graphs/titanexample/vertices/11111?name=zhangsan&age=24  #创建节点
    
curl -XPOST http://localhost:8182/graphs/titanexample/edges?_outV=<id>&_label=friend&_inV=2&<key>=<key'>  #创建节点间的边

  4. gremlin查询示例，查询贾宝玉的父亲

    gremlin> g.V.has('name','贾宝玉').next().out('father').name
    
==>贾政

  5. 全文检索节点,按照es的格式封装的map

    result = client.execute("g.indexQuery(\"vertices\", \"v.age:(17)\").vertices().get(0).getElement()");
    
for (Map<String, Object> map : result) {
    
    for (Map.Entry<String, Object> en : map.entrySet()) {
    
        System.out.print(en.getKey()+":"+en.getValue()+"\t\t");
    
    }
    
    System.out.println();
    
}
    
/**
    
_type:vertex		_properties:{name=薛宝钗, age=17}		_id:16496
    
*/

  6. 数据展现

    （1）Sigma.js it's a free and open source tool for graph visualization quite nice. Linkurious is using a fork version of it as far as I know in their product.

    （2）VivaGraph it's another free and open source tool for graph visualization tool - but it has a smaller community compared to SigmaJS.

    （3）D3.js it's the factotum for data visualization, you can do basically every kind of visualization based on that, but the learning curve is quite steep.

    （4）Gephi is another free and open source desktop solution, you have to use an external plugin with that probably but it does support most of the formats out there - graphML, CSV, Neo4J, etc...

五. 参考网站

thinkerpop

gremlin与sql对比查询语法

sparql-gremlin插件

DataStax给出的图形查询语言，包括sparql，GraphQL, Cypher, Gremlin