目的
将gensim输出的格式转化为numpy array格式,支持作为scikit-learn,tensorflow的输入
实施
使用nltk库的停用词和网上收集的资料整合成一份新的停用词表,用来过滤文档中的停用词,也去除了数字和特殊的标点符号,最后将所有字母转化为小写形式。
以下是原文:
Subject: Re: Candida(yeast) Bloom, Fact or Fiction
From: pchurch@swell.actrix.gen.nz (Pat Churchill)
Organization: Actrix Networks
Lines: 17
I am currently in the throes of a hay fever attack. SO who certainly
never reads Usenet, let alone Sci.med, said quite spontaneously "
There are a lot of mushrooms and toadstools out on the lawn at the
moment. Sure that's not your problem?"
Well, who knows? Or maybe it's the sourdough bread I bake?
After reading learned, semi-learned, possibly ignorant and downright
ludicrous stuff in this thread, I am about ready to believe anything