Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

时间:2023-03-09 19:04:14
Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

  week4

streaming data format

下面讲 data lakes

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

schema-on-read: 从数据源读取raw data 直接放到 data lake 里,然后再读到model里

schema-on-write: 传统模式,把raw data 经过处理后放到data warehouse里,此时已经是结构化的数据,然后直接load 出来

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

data lake summary

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

week5 - big data management

针对大数据,传统DBMS 需要提高的地方

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

some solutiion

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

from DBMS to BDMS

BDMS 应该具有的特征

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

BASE 就是基于CAP理论的

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

 

一些常用的BDMS及其优缺点

Redis: an enhanced key-value store

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

Aerospike: a new generation KV store

这是一个分布式NoSQL database + KV store.  是强一致性的

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

AsterixDB: a DBMS for semistructured data.  大家都知道的mongodb 以json 格式存储j数据, 这个Asterix 和 mongodb 类似. 它提供ACID保证.

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

Solr : Text data searching. 基于Lucene的

应该是一种search engine, 不知道和 ES 什么区别.

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

反向索引,至少要包含 doc id list, 也可以包含更多信息

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

除了full text search, 还有下面的功能

   Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)

Vertica:a columnar DBMS

  

  Coursera, Big Data 2, Modeling and Management Systems (week 4/5/6)