file_path = str(input('请输入文件路径：'))

最近研究GUI时发现了一个对用户更友好的操作，可以利用tkinter库来打开一个窗口，直接选择文件，具体代码及实现效果如下（其中，select_file()是选择文件，select_dierectory()是选择文件夹）

import tkinter as tk
from tkinter import filedialog

def select_file():
    root =tk.Tk()
    root.withdraw() #隐藏主窗口
    file_path = filedialog.askopenfilename()

    return file_path

def select_directory():
    root =tk.Tk()
    root.withdraw() #隐藏主窗口
    dir_path = filedialog.askdirectory()

    return dir_path

if __name__=='__main__':
    select_file()

StackOverFlow关于这个有个很好的回答：
https://stackoverflow.com/questions/53162/how-can-i-do-a-line-break-line-continuation-in-python

来源：xuan196 https://blog.csdn.net/xuan196/article/details/115127416

2、python处理中文乱码的问题：
2.1 将要处理的乱码对象设置 encoding = utf-8''

    response = requests.get(url=url, headers=headers)
    response.encoding = 'utf-8'

2.2 先设置encode的编码为iso-8859-1，再进行encoding的utf-8的设置

 # 通用处理中文乱码的解决方案
 img_name = img_name.encode('iso-8859-1').decode('gbk')

来源：Ctrl精 https://blog.csdn.net/qq_43468607/article/details/116154254?spm=1001.2101.3001.6650.6&utm_medium=distribute.pc_relevant.none-task-blog-2~default~BlogCommendFromBaidu~Rate-6-116154254-blog-120749614.pc_relevant_paycolumn_v3&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2~default~BlogCommendFromBaidu~Rate-6-116154254-blog-120749614.pc_relevant_paycolumn_v3&utm_relevant_index=10

通过以上两种方式我解决了最近遇到的所有了乱码问题，也感谢两篇文章的作者，分享出来共勉，侵删。

Python编程_1ed_Eric Matthes.pdf

change the working directory of anaconda

In the terminal, run

jupyter notebook --generate-config

Modify the config file and restart Anaconda Navigator:

Open the jupyter_notebook_config.py file in any suitable text editor and modify the “c.NotebookApp.notebook_dir” entry to point to the desired working directory. You will have to modify the “\” to “\” in your windows file path. Make sure to uncomment the line by removing the “#”.

Save the file and restart the Anaconda Navigator.

get current working directory

os.getcwd()

enumerate

• loop through the items

lst = ["app", "banana", "gig"]
for thing in lst:
    print(thing)

• index + item: use enumerate

lst = ["app", "banana", "gig"]
for idx, thing in enumerate(lst):
    print(idx)
    print(thing)

How to find a subset of a list

sublist = [i for i in list if i > x]

The summation of list

Similar to union_all

a = [1,2,3]
b = [3,3,4]
a+b
# [1, 2, 3, 3, 3, 4]

Differences of loc and iloc and []

• Difference between df['col_name'].values and df[['col_name']].values. The former gives a 1d array and the latter gives a 2d array

• loc[] is the same as [] in most of the times!!! But it is better to call it explicitly

• Avoid chain indexing!!! like Ax['s']['as']. It can be replaced by .loc['as','s']

• The way to index on column name and row number without chain indexing

df.loc[df.index[0], 'NAME']
# or
df.iloc[0, df.columns.get_loc("a")]

• loc is label-based, which means that we have to specify the name of the rows and columns that we need to filter out.
  • For example, let’s say we search for the rows whose index is 1, 2 or 100. We will not get the first, second or the hundredth row here. Instead, we will get the results only if the name of any index is 1, 2 or 100.

# select all rows with a condition
data.loc[data.age >= 15]
# select with multiple conditions
data.loc[(data.age >= 12) & (data.gender == 'M')]
# Select a range of rows using loc
#slice
data.loc[1:3]
# Using loc, we can also slice the Pandas dataframe over a range of indices. If the indices are not in the sorted order, it will select only the rows with index 1 and 3
# Select only required columns with a condition
data.loc[(data.age >= 12), ['city', 'gender']]
# update a column with condition
data.loc[(data.age >= 12), ['section']] = 'M'
# update multiple columns with condition
data.loc[(data.age >= 20), ['section', 'city']] = ['S','Pune']

# select a column
data.loc[['col_name']]

# select index + column
data.loc[data.age >= 12,'col_name']

• On the other hand, iloc is integer index-based. So here, we have to specify rows and columns by their integer index.

# select rows with indexes
data.iloc[[0,2]]
# select rows with particular indexes and particular columns
data.iloc[[0,2],[1,3]]
# select a range of rows
data.iloc[1:3]
# select a range of rows and columns
data.iloc[1:3,2:4]

How to slice series

# df_temp is of pandas.series object
df_temp = df_all.isnull().sum(axis=0)
df_temp[df_temp>0]

• Select a particular column

df['label']

Basic picture

# packages
import matplotlib.pyplot as plt
%matplotlib inline

df_train['label'].value_counts().plot(kind='bar')
# create fig in each sub graphs
fig = plt.figure(figsize=(18 ,10))

for idx, row in enumerate(images):
    ax = fig.add_subplot(2,3,idx + 1)
    ax.set_xticks([])
    ax.set_yticks([])
    pixels = df_train.iloc[row, 1:786].values.reshape((28,28))
    ax.imshow(pixels, cmap="gray")
    ax.set_title(df_train.iloc[row]['label'], fontsize = 24)

Difference index, array, list

Array / List

Lists and arrays are used in Python to store data(any data type- strings, integers etc), both can be indexed and iterated also. Difference between lists and arrays are the functions that you can perform on them like for example when you want to divide an array by 4, the result will be printed on request but in case of a list, python will throw an error message.

index

Index, on the other hand, is immutable
Index: Immutable ndarray implementing an ordered, sliceable set.

Properties

dataframe.sum

dataframe.sum(axis=0)
按照行对每列进行sum

drop / dropna / isna /fillna

• index of missing values in a particular column

idx_missing = df[column].isna()

• find the rows withour missing values

df[-idx_missing]
df.loc[-idx_missing]

• fill na with value

# fill na with No College
# inplace means apply the changes to the original dataframe and no output
# will be produced
nba["College"].fillna("No College", inplace = True) 
# fill na with mean
nba["College"].fillna(np.mean(nba["College"]), inplace = True) 
# method : Method is used if user doesn’t pass any value. Pandas has different methods bfill/ ffill which fills the place with value in the Previous/Back respectively.
nba["College"].fillna(method = "ffill", inplace = True)
# 用空值前面的值去填充它

Matrix operation

In Python we can solve the different matrix manipulations and operations. Numpy Module provides different methods for matrix operations.

• add() − add elements of two matrices.

• subtract() − subtract elements of two matrices.

• divide() − divide elements of two matrices.

• multiply() − multiply elements of two matrices.

• dot() − It performs matrix multiplication, does not element wise multiplication.

• sqrt() − square root of each element of matrix.

• sum(x,axis) − add to all the elements in matrix. Second argument is optional, it is used when we want to compute the column sum if axis is 0 and row sum if axis is 1.

• “T” − It performs transpose of the specified matrix.

import numpy
# Two matrices are initialized by value
x = numpy.array([[1, 2], [4, 5]])
y = numpy.array([[7, 8], [9, 10]])
#  add()is used to add matrices
print ("Addition of two matrices: ")
print (numpy.add(x,y))
# subtract()is used to subtract matrices
print ("Subtraction of two matrices : ")
print (numpy.subtract(x,y))
# divide()is used to divide matrices
print ("Matrix Division : ")
print (numpy.divide(x,y))
print ("Multiplication of two matrices: ")
print (numpy.multiply(x,y))
print ("The product of two matrices : ")
print (numpy.dot(x,y))
print ("square root is : ")
print (numpy.sqrt(x))
print ("The summation of elements : ")
print (numpy.sum(y))
print ("The column wise summation  : ")
print (numpy.sum(y,axis=0))
print ("The row wise summation: ")
print (numpy.sum(y,axis=1))
# using "T" to transpose the matrix
print ("Matrix transposition : ")
print (x.T)

lambda functions

x = lambda a: a+1
x = lambda a, b : a * b

# Apply lambda function in dataframe
df['Percent Growth'].apply(lambda x: x.replace('%', '')).astype('float')

reshape numpy array

numpy allow us to give one of new shape parameter as -1 (eg: (2,-1) or (-1,3) but not (-1, -1)). It simply means that it is an unknown dimension and we want numpy to figure it out. And numpy will figure this by looking at the 'length of the array and remaining dimensions' and making sure it satisfies the above mentioned criteria

String formats

The format() method formats the specified value(s) and insert them inside the string's placeholder.

The placeholder is defined using curly brackets: {}. Read more about the placeholders in the Placeholder section below.

The format() method returns the formatted string.

txt1 = "My name is {fname}, I'am {age}".format(fname = "John", age = 36)
txt2 = "My name is {0}, I'am {1}".format("John",36)
txt3 = "My name is {}, I'am {}".format("John",36)

Inside the placeholders you can add a formatting type to format the result

• :< Left aligns the result (within the available space)
• :> Right aligns the result (within the available space)
• :^ Center aligns the result (within the available space)
• := Places the sign to the left most position
• :+ Use a plus sign to indicate if the result is positive or negative
• :- Use a minus sign for negative values only
• : Use a space to insert an extra space before positive numbers (and a minus sign befor negative numbers)
• :, Use a comma as a thousand separator
• :_ Use a underscore as a thousand separator
• :b Binary format
• :c Converts the value into the corresponding unicode character
• :d Decimal format
• :e Scientific format, with a lower case e
• :E Scientific format, with an upper case E
• :f Fix point number format :.2f means 2 digits are preserved
• :F Fix point number format, in uppercase format (show inf and nan as INF and NAN)
• :g General format
• :G General format (using a upper case E for scientific notations)
• :o Octal format
• :x Hex format, lower case
• :X Hex format, upper case
• :n Number format
• :% Percentage format

Open a file

The available modes are:

The default mode is 'r' (open for reading text, synonym of 'rt'). Modes 'w+' and 'w+b' open and truncate the file (先清空). Modes 'r+' and 'r+b' open the file with no truncation.

As mentioned in the Overview, Python distinguishes between binary and text I/O. Files opened in binary mode (including 'b' in the mode argument) return contents as bytes objects without any decoding. In text mode (the default, or when 't' is included in the mode argument), the contents of the file are returned as str, the bytes having been first decoded using a platform-dependent encoding or using the specified encoding if given.

flatten

numpy.ndarray.flatten() function

The flatten() function is used to get a copy of an given array collapsed into one dimension.

‘C’ means to flatten in row-major (C-style) order. ‘F’ means to flatten in column-major (Fortran- style) order. ‘A’ means to flatten in column-major order if a is Fortran contiguous in memory, row-major order otherwise. ‘K’ means to flatten a in the order the elements occur in memory. The default is ‘C’.

ndarray.flatten(order='C')

underscore in python

Underscore _ is considered as "I don't Care" or "Throwaway" variable in Python

The underscore _ is also used for ignoring the specific values. If you don’t need the specific values or the values are not used, just assign the values to underscore.

x, _, y = (1, 2, 3)

>>> x
1

>>> y 
3

.copy()

df_copy = df_all

df_copy和df_all在这里会是联动的，只是称呼变了，就像vba里面的set一样

df_copy = df_all.pd.copy()

创造了一个新的object，两者是不联动的

Flatten a list

Given a list of l

flat_list = [item for sublist in l for item in sublist]

in in pandas

data.isin([])

继 Alonso 上篇用Python分析财险数据——菜鸟向，我对同样的数据用 Bokeh server 进行了可视化。

Bokeh简单介绍：Bokeh 是 Python 的一个制作交互式可视化工具的包，R 中也有相应的包叫做 shiny (https://shiny.rstudio.com/)。Boken 目前对中文的支持不太友好，但本文我们将用 JS 将网页语言改变为中文。Boken 有两种用法：

• 第一种是不利用 Bokeh server，这种情况下能做出好看的交互图，实现拖曳，放大，鼠标悬浮标签等功能。最后能够生成静态的HTML文件。
• 第二种是利用 Bokeh server，做一个 Web application。这种情况下能实现数据筛选调用等更多功能。一般使用 Flask + Bokeh，把 Bokeh 放置于 Flask application 里面。我们的这个例子中没有使用 FLask，而用了一个默认的 HTML 模板，叫做 Jinja，很多可以修改的功能被限制了。

本文介绍的是第二种，Bokeh server 的 Web application 应用示例，代码基于 Bokeh Gallery 里面的两个 sample。一个是 movie，一个是 crossfilter，链接见文末的参考文献。

先来示范一下效果：

• 筛选数据功能：

• 拖曳，选择，数据标签功能：

• 通过拖曳点的方式修改数据的功能：

该交互式图表目前 host 于http://49.234.103.189:5006/test 这个网页中。

步骤

安装 Bokeh

pure python 用户打开命令行：

pip install bokeh

conda 用户：

conda install bokeh

文件树

我们需要的文件树大概是这样的结构：

app 文件夹下有三个文件：一个是 main.py，是我们的 python 主文件；另一个是 templates 文件夹，里面放 index.html，是我们对于基本 html 框架的补充；还有一个是 lidata.csv，是我们的数据源文件。

分析数据

我们要根据公司，险种，险别来进行数据筛选，因此，我们首先要得到这几列有哪些情况。

# lidata就是Alonso的数据集
df_all = pd.read_csv(r'./app/data/lidata.CSV', header = 0)
# 计算ULR
df_all['ULR'] = df_all['UL'] / df_all['EP']
# 加入all是为了能够选择所有情况
unique_company = ["All"] + df_all['公司'].unique().tolist()
unique_business = ["All"] + df_all['险种'].unique().tolist() 
unique_product = ["All"] + df_all['险别'].unique().tolist() 

需要对不同险别展示不同颜色，代码如下

color = pl.mpl['Plasma'][len(unique_product)]
#这里Plasma是一个Bokeh自带的调色盘，帮助我们找到好看的配色
df_all["color"] = [color[unique_product.index(pro)] for pro in df_all["险别"].values]

我们还要筛选展示的事故年，因此，我们需要读取最小的事故年和最大的事故年。

year_start = df_all['事故年'].min()
year_end =  df_all['事故年'].max()

最后一个要准备的是要展示的数据y列是什么。这里需要做一个字典用来对应选项和数据列名的关系。

axis_map = {
    "ULR": "ULR",
    "ULAE": "EP",
    "DAC":'DAC'
}

接下来就是作图啦。图分为左右两边。左边的部分叫做 control，右边的部分叫做 plot。

制作control

# year_range: 展示的事故年范围
year_range = RangeSlider(start=year_start, end=year_end, value=(year_start,year_end), step=1,
                       title="展示年")
Slider(title="开始展示年", start=year_start, end=year_end, value=year_start, step=1)
max_year = Slider(title="结束展示年", start=year_start, end=year_end, value=year_end, step=1)
# 选择的公司，险别，险种
company = Select(title="公司选择", value="All",
               options=unique_company)
business = Select(title="险别选择", value="All",
               options=unique_business)
product = Select(title="险种选择", value="All",
               options=unique_product)
y_axis = Select(title="展示值", options=sorted(axis_map.keys()), value="ULR")

controls = [company, business, product, year_range,  y_axis]

制作plot

# Tooltips用来制作鼠标悬浮于数据时的数据标签
TOOLTIPS=[
    ("公司为", "@com"),
    ("年:", "@year"),
    ("险别为", "@business"),
    ("险种为", "@pro")
]
# TOOLS规定了哪些工具要显示出来，比如拖曳等
TOOLS="pan,wheel_zoom,box_select,lasso_select,reset"
p = figure(tools=TOOLS,plot_height=100, plot_width=200, title="", toolbar_location="above", tooltips=TOOLTIPS, sizing_mode="scale_both")
r = p.circle(x="x",y="y" ,source=source, size=10, color = 'color', alpha=0.6, hover_color='white', hover_alpha=0.5)
# PointDrawTool这个工具需要单独放入其中
draw_tool = PointDrawTool(renderers=[r], empty_value='black')
p.add_tools(draw_tool)
p.toolbar.active_tap = draw_tool

更新数据

def select_products():
    # strip可以去除数据前面或者后面的空格
    company_val = company.value.strip()
    business_val = business.value.strip()
    product_val = product.value.strip()
    # 选择事故年
    selected = df_all[
        (df_all.事故年 >= year_range.value[0]) &
        (df_all.事故年 <= year_range.value[1]) 
    ]
    # 选择公司，险种，险别等
    if (company_val != "All"):
        selected = selected[selected.公司.str.contains(company_val)==True]
    if (business_val != "All"):
        selected = selected[selected.险种.str.contains(business_val)==True]
    if (product_val != "All"):
        selected = selected[selected.险别.str.contains(product_val)==True]
    return selected

# 这个函数用来更新数据源
def update():
    df = select_products()
    x_name = "事故年"
    y_name = axis_map[y_axis.value]
    p.title.text = "%d points selected" % len(df)
    source.data = dict(
        x=df[x_name],
        y=df[y_name],
        com=df["公司"].values,
        year=df["事故年"].values,
        business=df["险别"].values,
        pro=df["险种"].values,
        color = df["color"]
    )
# control中的每一个元素改变后，都需要运行update()
for control in controls:
    control.on_change('value', lambda attr, old, new: update())

生成图

# input就是图左边的control
inputs = column(*controls, width=320, height=1000)
inputs.sizing_mode = "fixed"

l = layout([
    [inputs, p],
], sizing_mode="scale_both")

update()
curdoc().add_root(l, p)

templates文件夹：利用Bokeh自带Jinja模板对网页更改基本样式

这个时候就要用到templates文件夹啦！它里面的index.html是对于jinja模板的补充。

Jinja模板如下，这个我们没有办法改，想要改的话只能用JS在后面改。

<!DOCTYPE html>
<html lang="en">
{% block head %}
<head>
    {% block inner_head %}
    <meta charset="utf-8">
    <title>{% block title %}{{ title | e if title else "Bokeh Plot" }}{% endblock %}</title>
    {% block preamble %}{% endblock %}
    {% block resources %}
        {% block css_resources %}
        {{ bokeh_css | indent(8) if bokeh_css }}
        {% endblock %}
        {% block js_resources %}
        {{ bokeh_js | indent(8) if bokeh_js }}
        {% endblock %}
    {% endblock %}
    {% block postamble %}{% endblock %}
    {% endblock %}
</head>
{% endblock %}
{% block body %}
<body>
    {% block inner_body %}
    {% block contents %}
        {% for doc in docs %}
        {{ embed(doc) if doc.elementid }}
        {% for root in doc.roots %}
            {{ embed(root) | indent(10) }}
        {% endfor %}
        {% endfor %}
    {% endblock %}
    {{ plot_script | indent(8) }}
    {% endblock %}
</body>
{% endblock %}
</html>

index.html 的基本格式如下：

{% extends base %}

<!-- goes in head -->
{% block preamble %}
<link href="app/static/css/custom.min.css" rel="stylesheet">
{% endblock %}

<!-- goes in body -->
{% block contents %}
<div> {{ embed(roots.scatter) }} </div>
<div> {{ embed(roots.line) }} </div>
{% endblock %}

我在标准模板中加了一些代码，来保证 html 的语言选项是 zh，也就是中文，加以对CSS文件的修改，就大功告成啦！

window.onload = function() {
  document.querySelector("html").lang = "zh";
};

运行

在命令行中先 cd 到 app 所在文件夹，并输入:

bokeh serve app

或者

bokeh serve --show app

或在 Debug 模式运行

bokeh serve --log-level=debug app

当 python 由于版本不同可能有冲突时，可以使用:

python3 -m bokeh serve app

完整代码在github: https://github.com/Mengkee/bokeh_example

参考文献

Bokeh Gallery

Bokeh Sample: movies

Bokeh Sample: crossfilter

用Python分析财险数据——菜鸟向

放上本文用到的数据以供大家动手实验：

假数据.rar