Category: Python

ありたいてに言うと、下記のような構造のjsonをfor文で作りたい

{
	"item": [
		{
			"area": "東京都","population": 1300, "capital": "東京"
		},
		{
			"area": "北海道","population": 538, "capital": "札幌市"
		},
		{
			"area": "沖縄","population": 143, "capital": "那覇市"
		}	
	]
}

for文で辞書を配列に入れて、それを辞書の中に入れてjsonにすればOK

ys = cl.OrderedDict()
result = []
for item in data:
	print(item["at"][:16] +" " + item["anm"] + " " + item["mag"] + " " + item["maxi"] + " " + item["cod"][:5] + " " + item["cod"][5:11]+ " " + item["cod"][12:-1])
	data = cl.OrderedDict()
	data["at"] = item["at"][:16]
	data["anm"] = item["anm"]
	data["mag"] = item["mag"]
	data["maxi"] = item["maxi"]
	data["tokei"] = item["cod"][:5]
	data["hokui"] = item["cod"][5:11]
	data["depth"] = item["cod"][12:-1]

	result.append(data)

ys["item"] = result
print(ys)

with open("test.json", "w") as f:
    json.dump(ys, f, ensure_ascii=False)

pythonによるjson操作と辞書の理解が浅かったので、これ作るのに丸一日かかった　orz…
なんてこったい

import json

str = {
	"東京":{
		"population": 1300,
		"capital": "東京"
	},
	"北海道": {
		"population": 538,
		"capital": "札幌市"
	},
	"沖縄":{
		"population": 143,
		"capital": "那覇市"
	}
}

with open("population.json", "w") as f:
	json.dump(str, f, ensure_ascii=False)

{“東京”: {“population”: 1300, “capital”: “東京”}, “北海道”: {“population”: 538, “capital”: “札幌市”}, “沖縄”: {“population”: 143, “capital”: “那覇市”}}

なるほどー

ChatterBot:
The program selects the closest matching response by searching for the closest matching known statement that matches the input, it then returns the most likely response to that statement based on how frequently each response is issued by the people the bot communicates with.

chat.txt

こんにちは
こちらこそ こんにちは
今日もよろしくお願いします

chat2.txt

こんばんは
こちらこそ こんばんは
もう眠いですね
おやすみなさい

main.py

from chatterbot import ChatBot 
from chatterbot.trainers import ListTrainer

bot = ChatBot("チャットボット試作品")

training_ = open('chat.text','r').readlines()
training2_ = open('chat2.text','r').readlines()

trainer = ListTrainer(bot)

trainer.train(training_)
trainer.train(training2_)

while True:
	try:
		bot_input = bot.get_response(input())
		print(bot_input)
	except(KeyboardInterrupt, EOFError, SystemExit):
		break

$ pip3 install chatterbot
$ pip3 install chatterbot-corpus
$ python3 main.py
[nltk_data] Downloading package stopwords to
[nltk_data] /home/vagrant/nltk_data…
[nltk_data] Package stopwords is already up-to-date!
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data] /home/vagrant/nltk_data…
[nltk_data] Package averaged_perceptron_tagger is already up-to-
[nltk_data] date!
List Trainer: [####################] 100%
List Trainer: [####################] 100%
こんにちは
こちらこそ こんにちは
もう眠いですね
おやすみなさい
もう眠いですね
おやすみなさい
こちらこそ こんばんは
もう眠いですね
aaa
もう眠いですね
そうですか
こちらこそ こんばんは
おはよう
もう眠いですね
こんにちは
こちらこそ こんにちは

なんだこれ…

lavelImageをやり直します

import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "data/vision.json"

import io
import cv2
import matplotlib.pyplot as plt
import matplotlib
from google.cloud import vision
import xml.etree.ElementTree as ET

img = cv2.imread("img/license.jpeg")

client = vision.ImageAnnotatorClient()
with io.open("img/license.jpeg", 'rb') as image_file:
	content = image_file.read()
image = vision.Image(content=content)
response = client.document_text_detection(image=image)

text_infos = []
document = response.full_text_annotation
for page in document.pages:
  for block in page.blocks:
    for paragraph in block.paragraphs:
      for word in paragraph.words:
        for symbol in word.symbols:
          bounding_box = symbol.bounding_box
          xmin = bounding_box.vertices[0].x
          ymin = bounding_box.vertices[0].y
          xmax = bounding_box.vertices[2].x
          ymax = bounding_box.vertices[2].y
          xcenter = (xmin+xmax)/2
          ycenter = (ymin+ymax)/2
          text = symbol.text
          text_infos.append([text, xcenter, ycenter])

tree = ET.parse("data/license.xml")
root = tree.getroot()
result_dict = {}
for obj in root.findall("./object"):
  name = obj.find('name').text
  xmin = obj.find('bndbox').find('xmin').text
  ymin = obj.find('bndbox').find('ymin').text
  xmax = obj.find('bndbox').find('xmax').text
  ymax = obj.find('bndbox').find('ymax').text
  xmin, ymin, xmax, ymax = int(xmin), int(ymin), int(xmax), int(ymax)
  texts = ''
  for text_info in text_infos:
    text = text_info[0]
    xcenter = text_info[1]
    ycenter = text_info[2]
    if xmin <= xcenter <= xmax and ymin <= ycenter <= ymax:
      texts += text
  result_dict[name] = texts
  
for k, v in result_dict.items():
    print('{} : {}'.format(k, v))

$ python3 app.py
name : 日本花子
birthday : 昭和61年5月1日生)
address : 東京都千代田区霞加喂2-1-2
data of issue : 令和01年05月07日12345
expiration date : 12024年(今和06年)06月01日未有动
driver license : 運転免許証
number : 第012345678900号
Public Safety Commission : 00000公安委員会

Finally
やっと来たね
というか、サイズが一定なら免許証以外でもOCRできるね

$ sudo apt -y install tesseract-ocr tesseract-ocr-jpn libtesseract-dev libleptonica-dev tesseract-ocr-script-jpan tesseract-ocr-script-jpan-vert
$ tesseract -v
tesseract 4.1.3
leptonica-1.79.0
libgif 5.1.4 : libjpeg 8d (libjpeg-turbo 2.0.3) : libpng 1.6.37 : libtiff 4.1.0 : zlib 1.2.11 : libwebp 0.6.1 : libopenjp2 2.3.1
Found AVX2
Found AVX
Found SSE
Found libarchive 3.4.0 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4
$ tesseract –list-langs
List of available languages (5):
Japanese
Japanese_vert
eng
jpn
osd
※osdとは、言語の判定、文字角度の識別を行う

$ tesseract test1.png output
->
$ tesseract test1.png stdout
Hello world
$ tesseract test2.png stdout -l jpn
こんにちは

### Pythonで使えるようにする
$ pip3 install pyocr

from PIL import Image
import sys
import pyocr

tools = pyocr.get_available_tools()
langs = tools[0].get_available_languages()

img = Image.open('test1.png')
txt = tools[0].image_to_string(
	img,
	lang=langs[0],
	builder=pyocr.builders.TextBuilder(tesseract_layout=6)
)
print(txt)

※tesseract_layout=6 が精度に重要

$ python3 main.py
Hello world

lang=’jpn’に変更すると、日本語も出力できる
$ python3 main.py
とこんにちは
|
tesseract_layout=3で実行すると…
$ python3 main.py
こんにちは

#### tesseract_layoutの意味
0 = Orientation and script detection (OSD) only.
1 = Automatic page segmentation with OSD.
2 = Automatic page segmentation, but no OSD, or OCR
3 = Fully automatic page segmentation, but no OSD. (Default)
4 = Assume a single column of text of variable sizes.
5 = Assume a single uniform block of vertically aligned text.
6 = Assume a single uniform block of text.
7 = Treat the image as a single text line.
8 = Treat the image as a single word.
9 = Treat the image as a single word in a circle.
10 = Treat the image as a single character.

0 =方向とスクリプトの検出（OSD）のみ。
1 = OSDによる自動ページセグメンテーション。
2 =自動ページセグメンテーション、ただしOSDまたはOCRなし
3 =完全自動のページセグメンテーション。ただし、OSDはありません。 （ディフォルト）
4 =可変サイズのテキストの単一列を想定します。
5 =垂直に配置されたテキストの単一の均一なブロックを想定します。
6 =単一の均一なテキストブロックを想定します。
7 =画像を単一のテキスト行として扱います。
8 =画像を1つの単語として扱います。
9 =画像を円の中の1つの単語として扱います。
10 =画像を1文字として扱います。

6で設定されているのが多いですね
文字角度を考慮しなければ3で、文字角度を考慮する場合は6でしょうか？
読み取りたい文字に併せて設定を変えながら試していくイメージか…

from flask import Flask, render_template
import pymysql

app = Flask(__name__)

def getConnection():
	return pymysql.connect(
		host="localhost",
		db="hoge",
		user="fuga",
		password="hou",
		charset="utf8",
		cursorclass=pymysql.cursors.DictCursor
	)

@app.route("/")
def index():

	connection = getConnection()

	sql = "SELECT * FROM avalanche where id = 1"
	cursor = connection.cursor()
	cursor.execute(sql)
	results = cursor.fetchall()

	cursor.close()
	connection.close()
	return render_template('index.html', results=results)

if __name__ == "__main__":
	app.run(debug=True, host='0.0.0.0', port=5000)

なるほど
これをdockerでやりたい

mysql-connector-pythonでも良いが、pymysqlを使う場合

$ pip3 install pymysql

import pymysql

print("hello world")

connection = pymysql.connect(
	host="localhost",
	db="hoge",
	user="fuga",
	password="asdf",
	charset="utf8",
	cursorclass=pymysql.cursors.DictCursor
)

sql = "SELECT * FROM avalanche"
cursor = connection.cursor()
cursor.execute(sql)
posts = cursor.fetchall()

cursor.close()
connection.close()

for post in posts:
	print(post["title"])

ほう

sqlite3

create table results (
	id integer primary key autoincrement,
	title text,
	data text,
	img text,
	created datetime
);

models.py

def select(con, pk):
	cur = con.execute('select id, title, data, img, created from results where id=?', (pk,))
	return cur.fetchone()

def select_all(con):
	cur = con.execute('select id, title, data, img, created from results order by id desc')
	return cur.fetchall()

def delete(con, pk):
	cur = con.cursor()
	cur.execute('delete from results where id=?', (pk,))
	con.commit()

run.py

@app.route('/')
def index():
	con = get_db()
	results = models.select_all(con)
	return render_template('index.html', results=results)

@app.route('/delete/<pk>')
def delete(pk):
	con = get_db()
	models.delete(con, pk)
	return redirect(url_for('index'))

@app.route('/view/<pk>')
def view(pk):
	con = get_db()
	result = models.select(con, pk)
	return render_template("view.html", result=result)

index.html

    {% for result in results %}
    <tr>
        <td>{{result.id}}</td>
        <td>{{result.title|safe}}</td>
        <td>{{result.rcreated}}</td>
        <td>
            <a href="/view/{{result.id}}"><button class="btn btn-primary">参照</button></a>
            <form action="/delete/{{result.id}}" style="display:inline" method="post">
                <input class="btn btn-danger" type="submit" value="削除" onclick='return confirm("削除しますがよろしいですか")';>
            </form>
        </td>
    </tr>
    {% endfor %}

view.html

{% extends "base.html" %}
{% block body %}
<h1>結果参照</h1>

<h3>{{ result.id}}: {{ result.title|safe}}</h3>
<p>{{ result.created }}</p>
<div class="row">
	<img src="{{ url_for('static', filename=result.img)}}">
</div>

<div class="row">
	<textarea class="form-control" name="data" rows="5">{{result.data}}</textarea>
</div>
<br><br>
{% endblock %}

OK 一通り理解した
mysql & Dockerでやりたい

app.py

from flask import Flask
from flask import Flask, redirect, url_for, render_template

app = Flask(__name__)
app.config.update(dict(
	DATABASE = os.path.join(app.root_path, 'db.sqlite3'),
	SECRET_KEY='foo-baa',
))

@app.route('/')
def index():
	return render_template('index.html', result{})

@app.route('/create')
def create():
	return render_template('edit.html')

@app.route('/analysis', methods=['POST'])
def analysis():
	return redirect(url_for('view', pk=0))

@app.route('/delete/<pk>')
def delete(pk)
	return redirect(url_for('index'))

@app.route('/view/<pk>')
def view(pk):
	return render_template("index.html", values=values)

if __name__ == '__main__':
	app.run(debug=True, host='192.168.33.10', port=8000)

base.html

<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="UTF-8">
	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<title>簡易分析ツール</title>
	<script src="https://code.jquery.com/jquery-3.2.1.slim.min.js"
            integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN"
            crossorigin="anonymous"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js"
            integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q"
            crossorigin="anonymous"></script>
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js"
            integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl"
            crossorigin="anonymous"></script>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css"
          integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
    <link rel="stylesheet" type=text/css href="{{ url_for('static', filename='style.css') }}">
</head>
<body>

<header>
	<div class="navbar navbar-dark bg-dark box-shadow">
        <div class="container d-flex justify-content-between">
            <a href="/" class="navbar-brand d-flex align-items-center">
                <strong>簡易分析ツール</strong>
            </a>
        </div>
    </div>
</header>

<div class="container">
	{% block body %}
	{% endblock %}
</div>
</body>
</html>

edit.html

{% extends "base.html" %}
{% block body %}
<h1>新規分析</h1>
<form action="{{ url_for('analysis') }}">
	<label>タイトル</label>
	<input class="form-control" type="text" size="30" name="title">
	<label>分析データ</label>
	<textarea class="form-control" name="data" rows="5"></textarea>
	<input class="btn btn-primary" type="submit" value="送信">
</form>
{% endblock %}

view.html

{% extends "base.html" %}
{% block body %}
<h1>結果参照</h1>

<h3>0:サンプルタイトル</h3>
<p>2022/01/01 12:00</p>
<div class="row">
	ここにグラフを配置
</div>

<div class="row">
	<textarea class="form-control" name="data" rows="5">テストデータ</textarea>
</div>
<br><br>
{% endblock %}

http://192.168.33.10:8000/

ほう、これは凄い
Djangoに似てる