kubernetesとは?

kubernetesとは?
-> 自動デプロイ、スケーリング、アプリ・コンテナの運用自動化のために設計されたオープンソースのプラットフォーム
-> Dockerコンテナをクラスタ化した際の運用ツール

– アプリを迅速に予定通りにデプロイ(コンテナをサーバー群に展開)
– 稼働中にアプリをスケール(稼働中にコンテナ数を変更)
– 新機能をシームレスに提供開始(稼働中にロールアウト)
– ハードウェアの利用率を要求に制限(コンテナで共存させて稼働率を高くする)

パブリッククラウド、プライベートクラウド、ハイブリッドクラウド、マルチクラウド
モジュール化、追加可能、接続可能、構成可能
自動配置、自動再起動、自動複製、自動スケーリング

管理上の基本単位はPodという
– volume: 記憶領域を共有するコンテナの集まり
– Container
– IP ADress

要するに、
– アプリを迅速に予定通りにデプロイ
– 稼働中にアプリをスケールする
ことができるらしい。。。

WAYMO Open Dataset

流石に翻訳はないか。
https://waymo.com/open/

>The Dataset is about 1TB after compression for 1000 segments. Segments have been packaged into multiple files of about 25GB each.

>All segments contain 3D labels. The 100 segments that also contain 2D labels have been packaged into the first three files under ‘Training’ and the first file under ‘Validation’.

何これ?git hubもありますね。pythonです。
https://github.com/waymo-research/waymo-open-dataset/blob/master/tutorial/tutorial.ipynb

google colab
https://colab.research.google.com/notebooks/welcome.ipynb#recent=true

[vagrant@localhost local]$ python -V
Python 3.5.2

system requirement
– g++ 6 or higher.
– Python 3.
– TensorFlow 1.14.0 or higher.

g++って、c++じゃん。使わんなー

とりあえずgit clone
[vagrant@localhost waymo]$ git clone https://github.com/waymo-research/waymo-open-dataset.git waymo-od
Initialized empty Git repository in /home/vagrant/local/waymo/waymo-od/.git/
remote: Enumerating objects: 50, done.
remote: Counting objects: 100% (50/50), done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 174 (delta 17), reused 35 (delta 12), pack-reused 124
Receiving objects: 100% (174/174), 14.00 MiB | 2.93 MiB/s, done.
Resolving deltas: 100% (45/45), done.

[vagrant@localhost waymo-od]$ git checkout remotes/origin/r1.0
Note: checking out ‘remotes/origin/r1.0’.

You are in ‘detached HEAD’ state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b new_branch_name

HEAD is now at 2f30db6… Merge branch ‘master’ into r1.0
[vagrant@localhost waymo-od]$ git branch
* (no branch)
master

あれ?

sudo apt-get install --assume-yes pkg-config zip g++ zlib1g-dev unzip python3 python3-pip
BAZEL_VERSION=0.28.0
wget https://github.com/bazelbuild/bazel/releases/download/0.28.0/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh
sudo bash bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh
sudo apt install build-essential

これ、debian向けに書かれてる?

そして、bazelってビルドツール?makeでないんかい。

Googleが取り組む機械学習

Google, Amazon, Facebook, Apple, Microsoftはどのように機械学習に取り組んでいるのか?まずそれを知ることが大事。

■Google
自動運転への取り組みとしてWaymo
https://waymo.com/

続いて、Waymoのsoftware engineeringの採用情報を見てみましょう。

Data scientistの条件。phDか。。まじかー。
We’d like you to have:
PhD in statistics, math, or other quantitative area
Progressive statistics background either in academia or industry
Data science and system evaluation experience
Willingness to understand a complex system and its various components
Experience with tools for manipulating big data
Experience with R/Python and statistical libraries

続いてDeep Leaningのsoftware engineer
PhD in Computer Science, Machine Learning, Robotics, similar technical field of study, or equivalent practical experience
Experience in applied Machine Learning including data collection, analysis and feature engineering
Experience in Deep Learning research
Experience with TensorFlow
Experience programming in Python/C++

そうか、computer scienceはOKとして、Machine Learning, Roboticsは必須だな。

スタートアップとしては、プロトタイプを作りながら、ブラッシュアップが定説。
Hardwareの知識と組み合わせると、難易度が一気に上がる。つまりそこか。

まずはmachine learningとアプリケーションからか。

userの論理削除

論理削除用のカラムを追加します
[vagrant@localhost test]$ php artisan make:migration add_column_softDeletes_users_table –table=users
Created Migration: 2019_08_20_001651_add_column_softDeletes_users_table

migrationsにも追加されていますね。

laravel migrate:refreshとmigrate:resetの違い

migrate:refresh と migrate:reset 何が違うんや?
migrationを再実行の場合はrefreshでいいのかな?

[vagrant@localhost test]$ php artisan migrate:refresh
Nothing to rollback.

In Connection.php line 647:
                                                                                                                                   
  SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'users' already exists (SQL: create table `users` (`id` int unsi  
  gned not null auto_increment primary key, `name` varchar(255) not null, `email` varchar(255) not null, `password` varchar(255)   
  not null, `remember_token` varchar(100) null, `created_at` timestamp null, `updated_at` timestamp null) default character set u  
  tf8 collate utf8_unicode_ci)                                                                                                     
                                                                                                                                   

In Connection.php line 449:
                                                                                         
  SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'users' already exists

already existってエラーが出てる。resetでやってみると、

[vagrant@localhost test]$ php artisan migrate:reset
Nothing to rollback.                                                                                      

mysql> use test;
Database changed
mysql> show tables;
+—————-+
| Tables_in_test |
+—————-+
| migrations |
| users |
+—————-+
2 rows in set (0.00 sec)

あれ、うまくいってない??
mysql> drop table migrations;
Query OK, 0 rows affected (0.01 sec)

mysql> drop table users;
Query OK, 0 rows affected (0.00 sec)

[vagrant@localhost test]$ php artisan migrate
Migration table created successfully.
Migrating: 2014_10_12_000000_create_users_table
Migrated: 2014_10_12_000000_create_users_table
Migrating: 2014_10_12_100000_create_password_resets_table
Migrated: 2014_10_12_100000_create_password_resets_table

mysql> show tables;
+—————–+
| Tables_in_test |
+—————–+
| migrations |
| password_resets |
| users |
+—————–+
3 rows in set (0.00 sec)

いまいち理解できてないです。

公式では、、
migrate:refresh Reset and re-run all migrations
migrate:reset Rollback all database migrations
resethはrollbackで、refreshはre-runですな。

register controller

ユーザー情報はここで編集する

protected function validator(array $data)
    {
        return Validator::make($data, [
            'name' => ['required', 'string', 'max:255'],
            'email' => ['required', 'string', 'email', 'max:255', 'unique:users'],
            'password' => ['required', 'string', 'min:8', 'confirmed'],
        ]);
    }

    /**
     * Create a new user instance after a valid registration.
     *
     * @param  array  $data
     * @return \App\User
     */
    protected function create(array $data)
    {
        return User::create([
            'name' => $data['name'],
            'email' => $data['email'],
            'password' => Hash::make($data['password']),
        ]);
    }

Laravelの認証をカスタマイズする

/app/Http/Controllers/Auth/LoginController.php

namespace App\Http\Controllers\Auth;

use App\Http\Controllers\Controller;
use Illuminate\Foundation\Auth\AuthenticatesUsers;

class LoginController extends Controller
{
    /*
    |--------------------------------------------------------------------------
    | Login Controller
    |--------------------------------------------------------------------------
    |
    | This controller handles authenticating users for the application and
    | redirecting them to your home screen. The controller uses a trait
    | to conveniently provide its functionality to your applications.
    |
    */

    use AuthenticatesUsers;

    /**
     * Where to redirect users after login.
     *
     * @var string
     */
    protected $redirectTo = '/home';

    /**
     * Create a new controller instance.
     *
     * @return void
     */
    public function __construct()
    {
        $this->middleware('guest')->except('logout');
    }

    public function username()
    {
        return 'email';
    }
}

usernameメソッドを変更する

public function username()
    {
        return 'username';
    }

ほう、そういうことか。

laravelでlogin後のリダイレクト先を変更

App/Http/Middleware/RedirectAuthenticated.phpを編集します。

namespace App\Http\Middleware;

use Closure;
use Illuminate\Support\Facades\Auth;

class RedirectIfAuthenticated
{
    /**
     * Handle an incoming request.
     *
     * @param  \Illuminate\Http\Request  $request
     * @param  \Closure  $next
     * @param  string|null  $guard
     * @return mixed
     */
    public function handle($request, Closure $next, $guard = null)
    {
        if (Auth::guard($guard)->check()) {
            // return redirect('/home');
            return redirect('/top');
        }

        return $next($request);
    }
}

TopController.phpをつくります。

namespace App\Http\Controllers;

use Illuminate\Http\Request;

class TopController extends Controller
{
    /**
     * Create a new controller instance.
     *
     * @return void
     */
    public function __construct()
    {
        $this->middleware('auth');
    }

    /**
     * Show the application dashboard.
     *
     * @return \Illuminate\Contracts\Support\Renderable
     */
    public function index()
    {
        return view('top');
    }
}

上手くいくようになりました。

herokuで実行されない時

app logsを見ます。
[vagrant@localhost heroku]$ heroku logs -t
2019-08-17T06:29:07.714312+00:00 app[web.1]: [17-Aug-2019 06:29:07 UTC] PHP Fatal error: Uncaught Error: Call to undefined function mb_convert_encoding() in /app/backlog.php:23

ん? 
mb_convert_encoding()?

とりあえずcomposer.jsonをupdateして、herokuにpush

{
	"require" : {
		"ext-mbstring": "*",
		"google/apiclient": "^2.0"
	}
}

きたーーーーーーーーーーー 結局、GCP→Heroku→Google Spread sheetもOKでした。