Category: Python

from sklearn.datasets import load_boston
boston = load_boston()

import pandas as pd
boston_df = pd.DataFrame(boston.data, columns = boston.feature_names)
boston_df['MEDV'] = boston.target

boston_df.head()

TypeError: descriptor ‘__subclasses__’ of ‘type’ object needs an argument

[vagrant@localhost python]$ sudo yum install -y https://centos6.iuscommunity.org/ius-release.rpm

まず、python36を入れます。
[vagrant@localhost python]$ sudo yum install -y python36*

[vagrant@localhost python]$ pyenv versions
system
* 3.5.2 (set by /home/vagrant/.pyenv/version)
[vagrant@localhost python]$ pyenv install 3.6.4
Downloading Python-3.6.4.tar.xz…
-> https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tar.xz
Installing Python-3.6.4…
Installed Python-3.6.4 to /home/vagrant/.pyenv/versions/3.6.4

[vagrant@localhost python]$ pyenv global 3.6.4
[vagrant@localhost python]$ python -V
Python 3.6.4

[vagrant@localhost python]$ pip list
Package Version
——————– ——-
absl-py 0.8.0
astor 0.8.0
chainer 6.3.0
cycler 0.10.0
decorator 4.4.0
filelock 3.0.12
gast 0.2.2
google-pasta 0.1.7
grpcio 1.23.0
h5py 2.9.0
imageio 2.5.0
joblib 0.13.2
Keras 2.2.5
Keras-Applications 1.0.8
Keras-Preprocessing 1.1.0
kiwisolver 1.1.0
Markdown 3.1.1
matplotlib 3.1.1
networkx 2.3
numpy 1.17.1
pandas 0.25.1
Pillow 6.1.0
pip 19.2.3
protobuf 3.7.1
pyparsing 2.4.2
python-dateutil 2.8.0
pytz 2019.2
PyWavelets 1.0.3
PyYAML 5.1.2
scikit-image 0.15.0
scikit-learn 0.21.3
scipy 1.3.1
setuptools 28.8.0
six 1.12.0
tensorboard 1.14.0
tensorflow 1.14.0
tensorflow-estimator 1.14.0
termcolor 1.1.0
Theano 1.0.4
typing 3.6.6
typing-extensions 3.6.6
Werkzeug 0.15.5
wheel 0.33.6
wrapt 1.11.2

[vagrant@localhost python]$ python myapp.py
/home/vagrant/.pyenv/versions/3.6.4/lib/python3.6/site-packages/pandas/compat/__init__.py:84: UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
warnings.warn(msg)
/home/vagrant/.pyenv/versions/3.6.4/lib/python3.6/site-packages/pandas/compat/__init__.py:84: UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
warnings.warn(msg)

なにいいいいいいいいいいいいいいいいいいいいいい

The ast module makes it easy to handle Python abstract syntax trees in Python applications. The abstract syntax itself can change with every release of Python. Using this module will help to learn the current grammer programmatically.

To create an abstract syntax tree, pass ast. PyCF_ONLY_AST as a flag for the built-in function compile() or use the helper function parse() provided by this module. The result is a tree of objects of classes that inherit from ast.AST. Abstract syntax trees can be compiled into Python code objects using the built-in function compile().

[vagrant@localhost test]$ python --version
Python 3.5.2
[vagrant@localhost test]$ cat << 'EOF' > helloworld.py
> !#/usr/bin/env python
> # -*- coding: utf-8 -*-
>
> def main():
>   print('hello, world!')
>
> if __name__ == '__main__':
>   main()
> EOF

[vagrant@localhost test]$ python
Python 3.5.2 (default, Jul 28 2018, 11:25:01)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-23)] on linux
Type “help”, “copyright”, “credits” or “license” for more information.
>>> FILENAME = ‘helloworld.py’
>>> with open(FILENAME, ‘r’) as f:
… source = f.read()
File ““, line 2
source = f.read()
^
IndentationError: expected an indented block

あれ、うまくいかんな。。

from sklearn import svm, metrics
import random, re

lines = open('iris.csv', 'r', encoding='utf-8').read().split("\n")
f_tonum = lambda n : float(n) if re.match(r'^[0-9\.]+$', n) else n
f_cols = lambda li: list(map(f_tonum, li.strip().split(',')))
csv = list(map(f_cols, lines))
del csv[0]
random.shuffle(csv)

K = 5
csvk = [ [] for i in range(K) ]
for i in range(len(csv)):
	csvk[i % K].append(csv[i])

def split_data_label(rows):
	data = []; label = []
	for row in rows:
		data.append(row[0:4])
		label.append(row[4])
	return (data, label)

def calc_score(test, train):
	test_f, test_l = split_data_label(test)
	train_f, train_l = split_data_label(train)
	clf = svm.SVC()
	clf.fit(train_f, train_l)
	pre = clf.predict(test_f)
	return metrics.accuracy_score(test_l, pre)

score_list = []
for testc in csvk:
	trainc = []
	for i in csvk:
		if i != testc: trainc += i
	sc = calc_score(testc, trainc)
	score_list.append(sc)
print("各正解率=", score_list)
print("平均成果率=", sum(score_list) / len(score_list))

各正解率= [0.9666666666666667, 1.0, 1.0, 0.9333333333333333, 1.0]
平均成果率= 0.9800000000000001

import pandas as pd
from sklearn import cross_validation, svm, metrics
from sklearn.grid_search import GridSearchCV

train_csv = pd.read_csv("./mnist/train.csv")
test_csv = pd.read_csv("./mnist/t10k.csv")

train_label = train_csv.ix[:, 0]
train_data = train_csv.ix[:, 1:577]
test_label = test_csv.ix[:, 0]
test_data = test_csv.ix[:, 1:577]
print("学習データ数=", len(train_label))

params = [
	{"C": [1,10,100,1000], "kernel":["linear"]},
	{"C": [1,10,100,1000], "kernel":["rbf"], "gamma":[0.001, 0.0001]}
]

clf = GridSearchCV(svm.SVC(), params, n_jobs = -1)
clf.fit(train_data, train_label)
print("学習器=", clf.best_estimator_)

pre = clf.predict(test_data)
ac_score = metrics.accuracy_score(pre, test_label)
print("正解率=", ac_score)

import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn import cross_validation, metrics

mr = pd.read_csv("mushroom.csv", header=None)

label = []
data = []
attr_list = []
for row_index, row in mr.iterrows():
	label.append(row.ix[0])
	row_data = []
	for v in row.ix[1:]:
		row_data.append(ord(v))
	data.append(row_data)

data_train, data_test, label_train, label_test = \
	cross_validation.train_test_split(data, label)

clf = RandomForestClassifier()
clf.fit(data_train, label_train)

predict = clf.predict(data_test)

ac_score = metrics.accuracy_score(label_test, predict)
cl_report = metrics.classification_report(label_test, predict)
print("正解率=", ac_score)
print("レポート=\n", cl_report)

[vagrant@localhost python]$ python3 app.py
正解率= 1.0
レポート=
precision recall f1-score support

e 1.00 1.00 1.00 1031
p 1.00 1.00 1.00 1000

avg / total 1.00 1.00 1.00 2031

import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn import cross_validation, metrics

mr = pd.read_csv("mushroom.csv", header=None)

label = []
data = []
attr_list = []
for row_index, row in mr.iterrorws():
	label.append(row.ix[0])
	exdata = []
	for col, v in enumerate(row.ix[1:]):
		if row_index == 0:
			attr = {"dic": {}, "cnt":0}
			attr_list.append(attr)
		else:
			attr = attr_list[col]
		d = [0,0,0,0,0,0,0,0,0,0,0,0]
		if v in attr["dic"]:
			idx = attr["dic"][v]
		else:
			idx = attr["cnt"]
			attr["dic"][v] = idx
			attr["cnt"] += 1
			d[idx] = 1
			exdata += d
		data.append(exdata)

data_train, data_test, label_train, label_test = \
	cross_validation.train_test_split(data, label)

clf = RandomForestClassifier()
clf.fit(data_train, label_train)
predict = clf.predict(data_test)
ac_score = metrics.accuracy_score(label_test, predict)
print("正解率=", ac_score)