python 之金玉良言 或许是最后一次给自己系统总结--已结

jar tvf xxx.jar

vim xxx.jar （其实 底层调用 unzip , zip 命令)

配置一下 notepad++ F5

cmd /k D:"Program Files (x86)"\python\python.exe "$(FULL_CURRENT_PATH)" &pause &exit

# 查看已经连接过的 wifi 密码
for /f "skip=9 tokens=1,2 delims=:" %i in ('netsh wlan show profiles') do @echo %j | findstr -i -v echo | netsh wlan show profiles %j key=clear

进入 venv 生成 requirements.txt
pip freeze > requirements.txt

pip install -r requirements.txt

python 多重继承之拓扑排序

关键字：  DAG 有向无环图 ， 最左原则 实现 MRO (method resolution order) 方法解析顺序 之继承链

定制类

java 说我还是不太相信你，还是给你定一个规范吧， python 说 你是个成年人了，你知道为你的代码负责

不要对实例属性和类属性使用相同的名字，否则将产生难以发现的错误。实力属性总是会覆盖 类属型
getattr()、setattr()以及hasattr() ,dir() 内省

map filter 为 iter惰性计算的 Iterator, reduce 跟sorted 却是 立即计算

模块搜索路径
当我们试图加载一个模块时，Python会在指定的路径下搜索对应的.py文件，如果找不到，就会报错：

try:#python2
    from UserDict import UserDict
    #建议按照python3的名字进行import
    from UserDict import DictMixin as MutableMapping
except ImportError:#python3
    from collections import UserDict
    from collections import MutableMapping

>>> import mymodule
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named mymodule
默认情况下，Python解释器会搜索当前目录、所有已安装的内置模块和第三方模块，搜索路径存放在sys模块的path变量中：

>>> import sys
>>> sys.path
['', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6', ..., '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages']
如果我们要添加自己的搜索目录，有两种方法：

一是直接修改sys.path，添加要搜索的目录：

>>> import sys
>>> sys.path.append('/Users/michael/my_py_scripts')
这种方法是在运行时修改，运行结束后失效。

第二种方法是设置环境变量PYTHONPATH，该环境变量的内容会被自动添加到模块搜索路径中。设置方式与设置Path环境变量类似。注意只需要添加你自己的搜索路径，Python自己本身的搜索路径不受影响。

set和dict的唯一区别仅在于没有存储对应的value，但是，set的原理和dict一样，所以，同样不可以放入可变对象，因为无法判断两个可变对象是否相等，也就无法保证set内部“不会有重复元素”。

请务必注意，dict内部存放的顺序和key放入的顺序是没有关系的。

和list比较，dict有以下几个特点：

查找和插入的速度极快，不会随着key的增加而变慢；
需要占用大量的内存，内存浪费多。
而list相反：

查找和插入的时间随着元素的增加而增加；
占用空间小，浪费内存很少。
所以，dict是用空间来换取时间的一种方法。

dict可以用在需要高速查找的很多地方，在Python代码中几乎无处不在，正确使用dict非常重要，需要牢记的第一条就是dict的key必须是不可变对象。

这是因为dict根据key来计算value的存储位置，如果每次计算相同的key得出的结果不同，那dict内部就完全混乱了。这个通过key计算位置的算法称为哈希算法（Hash）。

要保证hash的正确性，作为key的对象就不能变。在Python中，字符串、整数等都是不可变的，因此，可以放心地作为key。而list是可变的，就不能作为key：

>>> key = [1, 2, 3]
>>> d[key] = 'a list'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'

l = [4,5,6]
t = (1,)
s = {1}
# s = set([1,2,3])
d = {'name':'frank'}

help(abs)

把函数作为参数传入，这样的函数称为高阶函数，函数式编程就是指这种高度抽象的编程范式。

from functools import reduce
import re
exp = '99 + 88 + 7.6 ='
reduce(lambda x,y:float(x)+float(y),
       filter(lambda x:True if x else False,
              map(lambda x:x.strip(),
                  re.split(r'[\+=]',exp))))

为什么要设计str、None这样的不变对象呢？因为不变对象一旦创建，对象内部的数据就不能修改，这样就减少了由于修改数据导致的错误。此外，由于对象不变，多任务环境下同时读取对象不需要加锁，同时读一点问题都没有。我们在编写程序时，如果可以设计一个不变对象，那就尽量设计成不变对象。

凡是可作用于for循环的对象都是Iterable类型；

凡是可作用于next()函数的对象都是Iterator类型，它们表示一个惰性计算的序列；

集合数据类型如list、dict、str等是Iterable但不是Iterator，不过可以通过iter()函数获得一个Iterator对象。

Python的for循环本质上就是通过不断调用next()函数实现的，例如：

for x in [1, 2, 3, 4, 5]:
    pass
实际上完全等价于：

# 首先获得Iterator对象:
it = iter([1, 2, 3, 4, 5])
# 循环:
while True:
    try:
        # 获得下一个值:
        x = next(it)
    except StopIteration:
        # 遇到StopIteration就退出循环
        break

"""
练习
小明身高1.75，体重80.5kg。请根据BMI公式（体重除以身高的平方）帮小明计算他的BMI指数，并根据BMI指数：

低于18.5：过轻
18.5-25：正常
25-28：过重
28-32：肥胖
高于32：严重肥胖
"""
height = 1.75
weight = 80.5

def BMI(height,weight):
    bmi = (weight/height)**2
    results = ("过轻","正常","过重","肥胖","严重肥胖")
    ret = results[-1]
    if bmi<18.5:
        ret = results[0]
    elif 18.5<=bmi<25:
        ret = results[1]
    elif 25<=bmi<28:
        ret = results[2]
    elif 28<=bmi<32:
        ret = results[3]
    else:
        pass
    return ret

BMI(height,weight)

def myhex(num):

    hex_list = []
    t_mt = ('A','B','C','D','E','F')
    while True:
        mod_, num = num %16, num//16
        hex_list.append(t_mt[mod_-10]) if mod_>9 else hex_list.append(str(mod_))
        if num== 0 :
            break
    hex_list.append('0x')
    hex_list.reverse()
    return ''.join(hex_list)

myhex(25)

def my_octonary(num):
    octonary_list = []
    while True:
        mod_,num = num%8,num//8
        octonary_list.append(str(mod_))
        if num == 0:
            break
    octonary_list.append('0o')
    octonary_list.reverse()
    return ''.join(octonary_list)

my_octonary(9)

# 基本递归
def fac(n):
    if n>1:
        return n*fac(n-1)
    elif n==1:
        return 1
    else:
        raise ValueError('数值必须大于等于1')

# 尾递归方式
def fac_iter(num,product):
    if num==1:
        return product
    else:
        return fac_iter(num-1,num*product)

# 循环实现
def fac2(n):
    if n<1:
        raise ValueError('数值必须大于等于1')
    acc = 1
    while n>=1:
        acc*=n
        n-=1
    return acc

fac_iter(4,1)

str2float

# -*- coding: utf-8 -*-
from functools import reduce
digital_dict = {'0':0,'1':1,'2':2,'3':3,'4':4,'5':5,'6':6,'7':7,'8':8,'9':9}
def str2float(s):
    i,j = s.split('.')
    left = reduce(lambda x,y:x*10+y,map(lambda x:digital_dict.get(x),i))
    right = reduce(lambda x,y:x/10+y,list(map(lambda x:digital_dict.get(x),j))[::-1])/10
    return left+right

str2float('123.456')

累乘

# -*- coding: utf-8 -*-
from functools import reduce
def prod(L):
    return reduce(lambda x,y:x*y,L)
print('3 * 5 * 7 * 9 =', prod([3, 5, 7, 9]))
if prod([3, 5, 7, 9]) == 945:
    print('测试成功!')
else:
    print('测试失败!')

名字这个我改了测试

names = ['adam', 'LISA', 'barT']

def normalize(names):
    return list(map(lambda name:str.upper(name[0])+str.lower(name[1:]),names))

# normalize(names)
# 测试:
L1 = ['adam', 'LISA', 'barT']
L2 = normalize(L1)
print(L2)

str2int

digital_dict = {'0':0,'1':1,'2':2,'3':3,'4':4,'5':5,'6':6,'7':7,'8':8,'9':9}
from functools import reduce

def str2int(s):
    return reduce(lambda x,y:x*10+y,map(lambda x:digital_dict.get(x),s))
str2int('13579')

def _odd_iter():
    n = 1
    while True:
        n = n + 2
        yield n

def _not_divisible(n):
    return lambda x: x % n > 0

def primes():
    yield 2
    it = _odd_iter() # 初始序列
    while True:
        n = next(it) # 返回序列的第一个数
        yield n
        it = filter(_not_divisible(n), it) # 构造新序列

# 打印1000以内的素数:
for n in primes():
    if n < 1000:
        print(n)
    else:
        break

def _odd_iter3():
    n = 3
    while True:
        yield n
        n+=2

def _not_divisible_3(n):
    return lambda x:x%n>0

def prime_iter3():
    yield 2
    it = _odd_iter()

    while True:
        base_num = next(it)
        yield base_num
        it = filter(lambda x,y=base_num:x%y>0,it)

for i in prime_iter3():
    if i>50:
        break
    else:
        print(i,end=',')

# -*- coding: utf-8 -*-

L = [('Bob', 75), ('Adam', 92), ('Bart', 66), ('Lisa', 88)]

def by_score(x):
    return x[1]

def by_name(x):
    return x[0]

sorted(L,key=by_score,reverse=True)
sorted(L,key=by_name,reverse=True)

def createCounter():
    count = 0
    def counter():
        nonlocal count
        count += 1
        return count
    return counter

def createCounter():
    def f():
        n=1
        while True:
            yield n
            n +=1
    g=f()
    def counter():
        return next(g)
    return counter

# 测试:
counterA = createCounter()
print(counterA(), counterA(), counterA(), counterA(), counterA()) # 1 2 3 4 5
counterB = createCounter()
if [counterB(), counterB(), counterB(), counterB()] == [1, 2, 3, 4]:
    print('测试通过!')
else:
    print('测试失败!')

def createCounter():
    x = 0
    def counter():
        nonlocal x
        x += 1
        return x
    return counter

from collections import Counter
Counter(s=3, c=2, e=1, u=1)
Counter({'s': 3, 'c': 2, 'u': 1, 'e': 1})
some_data=('c', '2', 2, 3, 5, 'c', 'd', 4, 5, 'd', 'd')
Counter(some_data).most_common(2)
[('d', 3), ('c', 2)]
some_data=['c', '2', 2, 3, 5, 'c', 'd', 4, 5, 'd', 'd']
Counter(some_data).most_common(2)
[('d', 3), ('c', 2)]
some_data={'c', '2', 2, 3, 5, 'c', 'd', 4, 5, 'd', 'd'}
Counter(some_data).most_common(2)
[('c', 1), (3, 1)]

class Fib():

    # 构造方法
    def __init__(self,a=0,b=1):
        self.__a,self.__b = a,b

    def __enter__(self):
        print('Begin')
        # 这里 实际可以新增 打开资源动作
        return self

    def __exit__(self,exc_type,exc_value,traceback):
        if exc_type:
            print('Error')
        else:
            print('End')

    #生成器 1
    def __iter__(self):
        return self

    # 生成器 2
    def __next__(self):
        self.__a, self.__b = self.__b, self.__a + self.__b

        if self.__a >10:
            raise StopIteration()

        return self.__a

    def __generatorfib(self,start,stop):
        x, y = 1,1
        print(start,'-->',stop)
        begin = int(start)
        end = int(stop)
        while begin<=end:
                yield x
                x, y = y,x+y
                begin, end = begin+1,end-1

    # 做成可下标取
    def __getitem__(self,n):
        if isinstance(n,int):
            x, y = 1,1
            for i in range(n):
                x, y = y,x+y
            return x

        # 下面这个没有真正支持 step
        if isinstance(n,slice):
            start = n.start if n.start else 0
            stop = n.stop
            step = n.step
            x,y = 1,1
            L = []
            for n in range(0,stop,step):
                if n>=start:
                    L.append(x)
                x, y = y,x+y
            return L[start:stop:step]

#         if isinstance(n,slice):
#             start = slice.start
#             stop = slice.stop
#             step = slice.step
#             inner_fib = self.__generatorfib(start,stop)
#             L = list(inner_fib)
#             return L[start,stop,step]

    def __getattr__(self,attr):
        if attr=='age':
            return lambda :18
        raise AttributeError('\'Fib\' object has no attribute \'{}\''.format(attr))

    # 打印给 user 看
    def __str__(self):
        return 'Fib({},{})'.format(self.__a,self.__b)

    # 打印给程序员看
    def __repr__(self):
        return self.__str__()
    # 重复声明一次是多余的但是指出这种用法
    __repr__ = __str__

for n in eval('Fib(0,1)'):
    print(n,end=',')

with Fib() as f:
    f[0:5:1]
    f.age()
    f.score()

元类 暂时学到这儿 ，产生 class 对象的 模板 必须继承自 type , 先有类来后有天，元类更在类之前, python 的 动态特性简直如此憨直 动到极致

class Field(object):

    def __init__(self,name,column_type):
        self.name = name
        self.column_type = column_type

    def __str__(self):
        return 'Field({name},{column_type})'.format(name=self.name,column_type=self.column_type)

    __repr__ = __str__

class IntegerField(Field):

    def __init__(self,name):
        super(IntegerField,self).__init__(name,'bigint')

class VarcharField(Field):

    def __init__(self,name):
        super(VarcharField,self).__init__(name,'varchar(256)')

class ModelMetaclass_frank(type):

    def __new__(cls,name,bases,attrs):
        if name=='Model':
            return type.__new__(cls,name,bases,attrs)
        print('found sub-model: {sub_model}'.format(sub_model=name))

        #准备一个 object 属性 与 列关系映射
        mappings = {}
        # 遍历 类属性仅 将 instance 为 Field  的 添加到 mappings 里保存起来
        for attr_key,attr_value in attrs.items():
            # attr_value ==》
            if isinstance(attr_value,Field):
                print('Found mapping: {}==>{}'.format(attr_key,attr_value))
                mappings[attr_key] = attr_value

        # 去除掉 attr (类属性) 中 有的，但是 mappings(可以看作是 实例属性 这里有点绕 这里的 实例是 type 创建出来的 class 实例) 中也有的 属性 ，避免 属性覆盖
        for instance_key in mappings.keys():
            attrs.pop(instance_key)

        # 将保存好的  实例属性 以及 table 名字 保留到 attr 中
        attrs['__mappings__'] = mappings
        attrs['__table__'] = str.lower(name)  # 这里的 那么 完全可以改为 xxx_+ name  比如 订单 register_user ,login_user
        return type.__new__(cls,name,bases,attrs)

class Model(dict,metaclass=ModelMetaclass_frank):

    def __init__(self,*args,**kw):
        super(Model, self).__init__(*args,**kw)

    def __getattr__(self,key):
        try:
            value = self.get(key)
        except KeyError as e:
            raise AttributeError(r'"Model" object has no attribute "{}"'.format(key))
        return value
#     def __setattr__(self,key,value):
#         self[key] = value

    def  save(self):
        fields = []
        params = []
        args = []

        for instance_attr_key,instance_attr_value in self.__mappings__.items():
            fields.append(instance_attr_value.name) # 这里是 Field 类中的 self.name
            params.append('?')
#             print('-->',getattr(self,instance_attr_key,None))
#             args.append(getattr(self,instance_attr_key,None))
            args.append(self.__getattr__(instance_attr_key))

        print('fiels: --> ',','.join(fields))
        print('params: -->',','.join(params))
        print('args: -->',args)
        args = list(map(lambda s: '\''+s+'\'' if isinstance(s,str) else str(s),args))
        sql = 'insert into {table_name}({fields_str}) values({values})'.format(table_name=self.__table__,fields_str=','.join(fields),values=','.join(args))
        print(sql)

class User(Model):
    # 定义类的属性到列的映射：
    id = IntegerField('id')
    name = VarcharField('username')
    email = VarcharField('email')
    password = VarcharField('password')

# 创建一个实例：
u = User(id=12345, name='Michael', email='[e-mail]', password='my-pwd')
u.save()

class Field(object):

    def __init__(self,name,column_type):
        self.name = name
        self.column_type = column_type

    def __str__(self):
        return ''.format(name=self.name,column_type=self.column_type)

class IntegerField(Field):

    def __init__(self,name):
        super(IntegerField,self).__init__(name,'bigint')

class StringField(Field):

    def __init__(self,name):
        return super(StringField,self).__init__(name,'varchar(256)')

# 元类必须继承自 type
class ModelMetaclass(type):

    def __init__(self,*args,**kw):
        return super(ModelMetaclass,self).__init__(*args,**kw)

    def __new__(cls,name,bases,attrs):
        print(cls,'<-->',name)
        if name=='Model':
            return type.__new__(cls,name,bases,attrs)
        print('Founding sub-model {sub_model}'.format(sub_model=name))

        mappings = {}
        for k,v in attrs.items():
            if isinstance(v,Field):
                mappings[k] = v

        for k in mappings.keys():
            attrs.pop(k)

        attrs['__mappings__'] = mappings
        attrs['__table__'] = attrs.get('__table__',str.lower(name))
        return type.__new__(cls,name,bases,attrs)

class Model(dict,metaclass=ModelMetaclass):

    def __init__(self,*args,**kw):
        super(Model,self).__init__(*args,**kw)

    def __getattr__(self,key):
        try:
            value = self.get(key)
        except KeyError:
            raise('\'{object}\' has no attr \'{attr}\''.format(object=self.__name__,attr=key))
        return value

    def save(self):
        fields = []
        args = []

        for k,v in self.__mappings__.items():
            fields.append(v.name)
            args.append(self.__getattr__(k))

        field_list = ','.join(fields)
        arg_list = ','.join(list(map(lambda x: '\'{}\''.format(x) if isinstance(x,str) else str(x),args)))
        sql = 'insert into {table_name}({field_list}) values({arg_list})'.format(table_name=self.__table__,field_list=field_list,arg_list=arg_list)
        print(sql)

class LoginUser(Model):
    id = IntegerField('id')
    name = StringField('name')
    email = StringField('email')
    password = StringField('password')

    __table__ = 'loginuser'

class RegisterUser(Model):
    id = IntegerField('id')
    name = StringField('name')
    email = StringField('email')
    password = StringField('password')

    __table__ = 'registeruser'

login_user = LoginUser(id=1,name='Frank',email='[e-mail]',password='passwordxxx')
login_user.save()

register_user = RegisterUser(id=1,name='Frank',email='[e-mail]',password='passwordxxx')
register_user.save()

内建模块

# datetime
from datetime import datetime,timedelta

now = datetime.now()

# datetime 转 timestamp
now_timestamp = now.timestamp()

# timestampe 转本地 datetime
dt_local = datetime.fromtimestamp(now_timestamp)
# timestampe 转utc datetime
dt_utc = datetime.utcfromtimestamp(now_timestamp)

# 时间戳 没有时区， datetime中携带
print(dt_local.timestamp(),'<-->',dt_utc.timestamp())

print('{}\n{}\n{}\n{}'.format(now,now_timestamp,dt_local,dt_utc))
# 获取指定 日期和时间
year = 2019
month =3
day =3
hour = 15
minute = 7
dt_specified = datetime(year,month,day,hour,minute)
print(dt_specified)

# str 转 datetime  str parse
datetime_str = '2019-03-03 15:22:00'
datetime_parse_format = '%Y-%m-%d %H:%M:%S'
cday = datetime.strptime(datetime_str,datetime_parse_format)
print(cday)

# datetime 转 str  str format
print(cday.strftime('%Y/%m/%d'))

# 日期变化（delta） 用 timedelta
now = datetime.now()
now_next3_hours =  now+timedelta(hours=3)
now_previous3_days = now+timedelta(days=-3)
print('next 3 hours: {}'.format(now_next3_hours))

print('now_previous3_days: {}'.format(now_previous3_days))

from datetime import timezone

tz_utc_8 = timezone(timedelta(hours=8))
now = datetime.now()
# 一开始 now 时区信息为 None
print(now.tzinfo)
# 暴力设置一个时区
now.replace(tzinfo=tz_utc_8)
print(now)

utc_now = datetime.utcnow()
# 一开始这玩意儿压根木有时区信息啊
print(utc_now.tzinfo)
# 暴力设置时区信息
utc_now = utc_now.replace(tzinfo=timezone.utc)

#北京日期时间 东八区
bj_dt = utc_now.astimezone(timezone(timedelta(hours=8)))
# 西八区
pst_dt = utc_now.astimezone(timezone(timedelta(hours=-8)))
# 东 9 区
tokyo_dt = utc_now.astimezone(timezone(timedelta(hours=9)))

print('bj_dt: ',bj_dt)
print('pst_dt: ',pst_dt)
print('tokyo_dt: ',tokyo_dt)

from datetime import datetime, timezone,timedelta
import re

def to_timestamp(dt_str,tz_str):
    re_dt_str_1 = r'\d{4}-\d{1,2}-\d{1,2}\s\d{1,2}:\d{1,2}:\d{1,2}'

    re_tz_str = r'^UTC([+-])(\d{1,2}):\d{2}$'

    tz_grps = re.match(re_tz_str,tz_str).groups()

    sign = tz_grps[0]
    hours = int(tz_grps[1])

    if re.match(re_dt_str_1,dt_str):
        dt = datetime.strptime(dt_str,'%Y-%m-%d %H:%M:%S')
        if sign=='+':
            tz_info_x = timezone(timedelta(hours=hours))
        else:
            tz_info_x = timezone(timedelta(hours=-hours))
        dt = dt.replace(tzinfo=tz_info_x)
    else:
        print('re is wrong!')

    return dt.timestamp()

# 测试:
t1 = to_timestamp('2015-6-1 08:10:30', 'UTC+7:00')

assert t1 == 1433121030.0, t1

t2 = to_timestamp('2015-5-31 16:10:30', 'UTC-09:00')
assert t2 == 1433121030.0, t2

print('ok')

collections 模块

def chunks(l, n):
    return [l[i:i+n] for i in range(0, len(l), n)]

chunks(s,2)

from collections import Iterator, Iterable
from collections import defaultdict
from collections import Counter, ChainMap, OrderedDict, namedtuple, deque
from itertools import islice  #  替代 切片，但是只能 是正数
from itertools import zip_longest # 替代 zip 可以 对不一样个数的 进行迭代

from concurrent.futures import ThreadPoolExecutor as Pool

from collections import namedtuple, deque, defaultdict, OrderedDict, ChainMap, Counter

Point = namedtuple('Poing',['x','y','z'])
p = Point(1,2,3)
print(p.x,'--',p.y,'--',p.z)

# 双向列表
dq = deque([1,2,3,4])
dq.append(5)
dq.appendleft('a')
dq.popleft()

default_dict = defaultdict(lambda:'N/A') # 多了一个默认值
default_dict['name']='frank'
default_dict['age']

od = OrderedDict([('b',1),('a',2),('c',3)]) # 按照插入的顺序有序
od.get('a')

# 可以实现一个FIFO（先进先出）的dict，当容量超出限制时，先删除最早添加的Key
from collections import OrderedDict

class LastUpdatedOrderedDict(OrderedDict):

    def __init__(self, capacity):
        super(LastUpdatedOrderedDict, self).__init__()
        self._capacity = capacity

    def __setitem__(self, key, value):
        containsKey = 1 if key in self else 0
        if len(self) - containsKey >= self._capacity:
            last = self.popitem(last=False)
            print('remove:', last)
        if containsKey:
            del self[key]
            print('set:', (key, value))
        else:
            print('add:', (key, value))
        OrderedDict.__setitem__(self, key, value)

# 应用场景 设置参数优先级
from collections import ChainMap
import os, argparse

# 构造缺省参数:
defaults = {
    'color': 'red',
    'user': 'guest'
}

# 构造命令行参数:
parser = argparse.ArgumentParser()
parser.add_argument('-u', '--user')
parser.add_argument('-c', '--color')
namespace = parser.parse_args()
command_line_args = { k: v for k, v in vars(namespace).items() if v }

# 组合成ChainMap:
combined = ChainMap(command_line_args, os.environ, defaults)

# 打印参数:
print('color=%s' % combined['color'])
print('user=%s' % combined['user'])

c = Counter()
for ch in 'programing':
    c[ch] +=1

print(c)

c = dict()
for ch in 'programing':
    if ch in c:
        c[ch] +=1
    else:
        c[ch] = 1

给单词按首字母分组
方法一:
words = ['apple','bat','bar','atom','book']
by_letter_grp_dict = {}
for word in words:
    letter = word[0]
    if letter in by_letter_grp_dict:
        by_letter_grp_dict[letter].append(word)
    else:
        by_letter_grp_dict[letter] = [word]
print(by_letter_grp_dict)
方法二:
内置的collections模块有一个叫做defaultdict的类，它可以使该过程更简单。传入一个类型或函数（用于生成字典各插槽所使用的默认值）即可创建一个defaultdict：

words = ['apple','bat','bar','atom','book']
from collections import defaultdict
by_letter = defaultdict(list)
for word in words:
    by_letter[word[0]].append(word)
print by_letter
#defaultdict(<type 'list'>, {'a': ['apple', 'atom'], 'b': ['bat', 'bar', 'book']})
计数使用collections 库的 Counter
>>> from collections import Counter
>>> c = Counter('hello world!')
>>> c
Counter({'l': 3, 'o': 2, 'h': 1, 'e': 1, ' ': 1, 'w': 1, 'r': 1, 'd': 1, '!': 1})
>>> c.most_common(1)
[('l', 3)]
>>> c.most_common(3)
[('l', 3), ('o', 2), ('h', 1)]
>>>

>>> import collections
>>> Card = collections.namedtuple('Card',['rank','suit'])
>>> class French_Deck(object):
...     rank = [ str(i) for i in range(2,11,1)] + list('JQKA')
...     suit = 'Spade,Club,Heart,Diamond'.split(',')
...     def __init__(self):
...         self._cards = [Card(r,s) for s in suit for r in rank]
...     def __getitem__(self,position):
...         return self._cards[position]
...     def __len__(self):
...         return len(self._cards)
...
>>> cards = French_Deck()
>>> len(cards)
56

from collections import namedtuple
stock_list = [['AAPL','10.30','11.90'],['YAHO','9.23','8.19'],['SINA','22.80','25.80']]
stock_info = namedtuple('stock_info',['name','start','end'])
[stock_info(name,start,end) for name,start,end in stock_list ]
stock_list

# 与 update 不一样哦。。。

from collections import ChainMap

dict_1 = {'name':'Frank','age':18}
dict_2 = {'name':'','age':20}
dict_coalesce = ChainMap(dict_1,dict_2)
dict_coalesce['name']

itertools 模块

from itertools import count, cycle, repeat, takewhile , chain, groupby

for i in count(1):
    if i<=10:
        print(i,end=',')
    else:
        break

print('\n'+'*'*100+'\n')

nums = takewhile(lambda n:n<=10,count(1))
print(list(nums))

for nl in repeat('\n',3):
    print(nl)

c = 0
for ch in cycle('ABC'):
    if c>=10:
        break
    else:
        print(ch,end=',')
    c+=1

print('\n'+'*'*100+'\n')

for ch in chain('ABC','XYZ'):
    print(ch)

for key,group in groupby('AAAaaabbBBBBBBBCCCccccddddEEEEDDD',lambda ch:ch.upper()):
    print(key,'--> ',list(group))

# -*- coding: utf-8 -*-
import itertools
from functools import reduce
def pi(N):
    ' 计算pi的值 '
    # step 1: 创建一个奇数序列: 1, 3, 5, 7, 9, ...
    odd_iter = itertools.count(1,2)

    # step 2: 取该序列的前N项: 1, 3, 5, 7, 9, ..., 2*N-1.
    odd_head = itertools.takewhile(lambda n:n<=2*N-1,odd_iter)
#     print(list(odd_head),end=',')
    # step 3: 添加正负符号并用4除: 4/1, -4/3, 4/5, -4/7, 4/9, ...
    odd_final = [4/n * ((-1)**i) for i,n in enumerate(odd_head)]
    # step 4: 求和:
    value = reduce(lambda x,y:x+y,odd_final)
    return value
# 测试:
print(pi(10))
print(pi(100))
print(pi(1000))
print(pi(10000))
assert 3.04 < pi(10) < 3.05
assert 3.13 < pi(100) < 3.14
assert 3.140 < pi(1000) < 3.141
assert 3.1414 < pi(10000) < 3.1415
print('ok')

contextlib 模块

from contextlib import contextmanager,closing

xml.parsers.expat , lxml 等 模块

import lxml
from xml.parsers.expat import ParserCreate

class DefaultSaxHandler(object):
    def start_element(self, name, attrs):
        print('sax:start_element: %s, attrs: %s' % (name, str(attrs)))

    def end_element(self, name):
        print('sax:end_element: %s' % name)

    def char_data(self, text):
        print('sax:char_data: %s' % text)

xml = r'''<?xml version="1.0"?>
<ol>
    <li><a href="/python">Python</a></li>
    <li><a href="/ruby">Ruby</a></li>
</ol>
'''

handler = DefaultSaxHandler()
parser = ParserCreate()
parser.StartElementHandler = handler.start_element
parser.EndElementHandler = handler.end_element
parser.CharacterDataHandler = handler.char_data
parser.Parse(xml)

html 解析模块 html

from html.parser import HTMLParser
from html.entities import name2codepoint

class MyHTMLParser(HTMLParser):

    def handle_starttag(self, tag, attrs):
        print('<%s>' % tag)

    def handle_endtag(self, tag):
        print('</%s>' % tag)

    def handle_startendtag(self, tag, attrs):
        print('<%s/>' % tag)

    def handle_data(self, data):
        print(data)

    def handle_comment(self, data):
        print('<!--', data, '-->')

    def handle_entityref(self, name):
        print('&%s;' % name)

    def handle_charref(self, name):
        print('&#%s;' % name)

parser = MyHTMLParser()
parser.feed('''<html>
<head></head>
<body>
<!-- test html parser -->
    <p>Some <a href=\"#\">html</a> HTML&nbsp;tutorial...<br>END</p>
</body></html>''')

---

第三方 字符编码检测模块 chardet

import chardet
data = '最新の主要ニュース'.encode('euc-jp')
chardet.detect(data)
{'confidence': 0.99, 'encoding': 'EUC-JP', 'language': 'Japanese'}

其他第三方模块

青出于蓝的 requests >> urllib
Pillow(新)  PIL(2.7 远古时代)
psutils  <== process and system utilities

将字典的值转成object 的属性，递归
d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}

def my_dict2obj(args):
    class obj(object):
        def __init__(self,d):
            for key,value in d.items():
                if not isinstance(value,(list,tuple)):
                    setattr(self,key,obj(value) if isinstance(value,dict) else value)
                else:
                    setattr(self,key,[obj(i) if isinstance(i,dict) else i for i in value])
    return obj(args)

x = my_dict2obj(d)

print(x.d)

扩展拆箱
>>> a = [1,2,3,4,5,6]
>>> b,*c,d,e = a
>>> b
1
>>> c
[2, 3, 4]
>>> d
5
>>> e
6

给单词按首字母分组
方法一:
words = ['apple','bat','bar','atom','book']
by_letter_grp_dict = {}
for word in words:
    letter = word[0]
    if letter in by_letter_grp_dict:
        by_letter_grp_dict[letter].append(word)
    else:
        by_letter_grp_dict[letter] = [word]
print(by_letter_grp_dict)
方法二:
内置的collections模块有一个叫做defaultdict的类，它可以使该过程更简单。传入一个类型或函数（用于生成字典各插槽所使用的默认值）即可创建一个defaultdict：

words = ['apple','bat','bar','atom','book']
from collections import defaultdict
by_letter = defaultdict(list)
for word in words:
    by_letter[word[0]].append(word)
print by_letter
#defaultdict(<type 'list'>, {'a': ['apple', 'atom'], 'b': ['bat', 'bar', 'book']})
计数使用collections 库的 Counter
>>> from collections import Counter
>>> c = Counter('hello world!')
>>> c
Counter({'l': 3, 'o': 2, 'h': 1, 'e': 1, ' ': 1, 'w': 1, 'r': 1, 'd': 1, '!': 1})
>>> c.most_common(1)
[('l', 3)]
>>> c.most_common(3)
[('l', 3), ('o', 2), ('h', 1)]
>>>

我见青山多巍峨,料青山见我应如是

# -*- coding: utf-8 -*-
__author__ = 'Frank Li'
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server.bind(('127.0.0.1',6666))
clients = set()
print('server bind 127.0.0.1:6666...')

while 1:
    try:
        data,addr = server.recvfrom(1024)
        clients.add(addr)
        if not data or data.decode('utf-8')=='pong':
            continue
        print('%s:%s >>> %s' % (addr[0],addr[1],data.decode('utf-8')))
        for usr in clients:
            if usr!=addr:
                server.sendto(('%s:%s >>> %s' % (addr[0],addr[1],data.decode('utf-8'))).encode('utf-8'),usr)
    except Exception as e:
        pass

# -*- coding: utf-8 -*-
__author__ = 'Frank Li'

import socket,threading,os

client = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
client.sendto(b'pong',('127.0.0.1',6666))

def myinput():
    while 1:
        try:
            msg = input('>>>')
            yield msg
        except Exception as e:
            os._exit(0)

def getMsg(client):
    while 1:
        try:
            r = client.recv(1024)
            print('\n',r.decode('utf-8'),'\n>>>',end='')
        except Exception as e:
            pass

c = myinput()
def sendMsg(msg):
    while 1:
        msg = next(c)
        client.sendto(msg.encode('utf-8'),('127.0.0.1',6666))

threading.Thread(target=sendMsg,args=(client,)).start()
threading.Thread(target=getMsg,args=(client,)).start()

inspect

import inspect

def a(a, b=0, *c, d, e=1, **f):
    pass

aa = inspect.signature(a)
print("inspect.signature(fn)是:%s" % aa)
print("inspect.signature(fn)的类型：%s" % (type(aa)))
print("\n")

bb = aa.parameters
print("signature.paramerters属性是:%s" % bb)
print("ignature.paramerters属性的类型是%s" % type(bb))
print("\n")

for cc, dd in bb.items():
    print("mappingproxy.items()返回的两个值分别是：%s和%s" % (cc, dd))
    print("mappingproxy.items()返回的两个值的类型分别是：%s和%s" % (type(cc), type(dd)))
    print("\n")
    ee = dd.kind
    print("Parameter.kind属性是:%s" % ee)
    print("Parameter.kind属性的类型是:%s" % type(ee))
    print("\n")
    gg = dd.default
    print("Parameter.default的值是: %s" % gg)
    print("Parameter.default的属性是: %s" % type(gg))
    print("\n")

ff = inspect.Parameter.KEYWORD_ONLY
print("inspect.Parameter.KEYWORD_ONLY的值是:%s" % ff)
print("inspect.Parameter.KEYWORD_ONLY的类型是:%s" % type(ff))

import inspect

def func_a(arg_a, *args, arg_b='hello', **kwargs):
    print(arg_a, arg_b, args, kwargs)

if __name__ == '__main__':

    # 获取函数签名
    func_signature = inspect.signature(func_a)
    func_args = []
    # 获取函数所有参数
    for k, v in func_signature.parameters.items():
        # 获取函数参数后，需要判断参数类型
        # 当kind为 POSITIONAL_OR_KEYWORD，说明在这个参数之前没有任何类似*args的参数，那这个函数可以通过参数位置或者参数关键字进行调用
        # 这两种参数要另外做判断
        if str(v.kind) in ('POSITIONAL_OR_KEYWORD', 'KEYWORD_ONLY'):
            # 通过v.default可以获取到参数的默认值
            # 如果参数没有默认值，则default的值为：class inspect_empty
            # 所以通过v.default的__name__ 来判断是不是_empty 如果是_empty代表没有默认值
            # 同时，因为类本身是type类的实例，所以使用isinstance判断是不是type类的实例
            if isinstance(v.default, type) and v.default.__name__ == '_empty':
                func_args.append({k: None})
            else:
                func_args.append({k: v.default})
        # 当kind为 VAR_POSITIONAL时，说明参数是类似*args
        elif str(v.kind) == 'VAR_POSITIONAL':
            args_list = []
            func_args.append(args_list)
        # 当kind为 VAR_KEYWORD时，说明参数是类似**kwargs
        elif str(v.kind) == 'VAR_KEYWORD':
            args_dict = {}
            func_args.append(args_dict)

    print(func_args)