set -e とはコマンドの実行結果がエラー(= 終了ステータスが 0 以外)になった時にシェルスクリプトを自動的に中断させる機能
条件文(&& や || を含む)と共に使うシェル関数は set -e の効果が無効になる
contrib/devtools/split-debug.sh.in
set -e if [ $# -ne 3 ]; then echo "usage: $0 <input> <stripped-binary> <debug-binary>" fi
ソフトウェアエンジニアの技術ブログ:Software engineer tech blog
随机应变 ABCD: Always Be Coding and … : хороший
set -e とはコマンドの実行結果がエラー(= 終了ステータスが 0 以外)になった時にシェルスクリプトを自動的に中断させる機能
条件文(&& や || を含む)と共に使うシェル関数は set -e の効果が無効になる
contrib/devtools/split-debug.sh.in
set -e if [ $# -ne 3 ]; then echo "usage: $0 <input> <stripped-binary> <debug-binary>" fi
自分の能力を超えて仮想的な記憶領域を得ること
容量不足時にSSD/HDDなどに書き出す
e.g. 使用していないときはSSDに保存しておいて、使用時に取り出すなど
devtools/security-check.py
L R, W, Xの意味がよくわかりませんね。
L ARCH.PPC64 はPowerPCのことのように見えるが何をやっているか不明
def check_ELF_separate_code(binary): ''' Check that sections are appropriately separated in virtual memory, based on their permissions. This checks for missing -Wl,-z,separate-code and potentially other problems. ''' R = lief.ELF.SEGMENT_FLAGS.R W = lief.ELF.SEGMENT_FLAGS.W E = lief.ELF.SEGMENT_FLAGS.X EXPECTED_FLAGS = { # Read + execute '.init': R | E, '.plt': R | E, '.plt.got': R | E, '.plt.sec': R | E, '.text': R | E, '.fini': R | E, # Read-only data '.interp': R, '.note.gnu.property': R, '.note.gnu.build-id': R, '.note.ABI-tag': R, '.gnu.hash': R, '.dynsym': R, '.dynstr': R, '.gnu.version': R, '.gnu.version_r': R, '.rela.dyn': R, '.rela.plt': R, '.rodata': R, '.eh_frame_hdr': R, '.eh_frame': R, '.qtmetadata': R, '.gcc_except_table': R, '.stapsdt.base': R, # Writable data '.init_array': R | W, '.fini_array': R | W, '.dynamic': R | W, '.got': R | W, '.data': R | W, '.bss': R | W, } if binary.header.machine_type == lief.ELF.ARCH.PPC64: # .plt is RW on ppc64 even with separate-code EXPECTED_FLAGS['.plt'] = R | W # For all LOAD program headers get mapping to the list of sections, # and for each section, remember the flags of the associated program header. flags_per_section = {} for segment in binary.segments: if segment.type == lief.ELF.SEGMENT_TYPES.LOAD: for section in segment.sections: flags_per_section[section.name] = segment.flags # Spot-check ELF LOAD program header flags per section # If these sections exist, check them against the expected R/W/E flags for (section, flags) in flags_per_section.items(): if section in EXPECTED_FLAGS: if int(EXPECTED_FLAGS[section]) != int(flags): return False return True
C++におけるファイルの分割はクラス定義単位で行う
### ヘッダファイル
マクロ定義、型定義、プロトタイプ宣言の3つを書くのが基本
C++ではクラスを利用するには、そのクラスが提供するヘッダファイルを取り込む必要がある
以下のようなファイルがあったとする
class POS { private: int x; int y; public: POS(); int setPos(int tmpx, int tmpy); } POS::POS(){ x = 0; y = 0; } int POS::setPos(int tmpx, int tmpy){ x = tmpx; y = tmpy; return 0; }
これを分割する場合
POS.h
#ifndef POS_H #define POS_H class POS { private: int x; int y; public: POS(); int setPos(int tmpx, int tmpy); } #endif
POS.cpp
#include "POS.h" POS::POS(){ x = 0; y = 0; } int POS::setPos(int tmpx, int tmpy){ x = tmpx; y = tmpy; return 0; }
つまり、最初にheader情報を見て、全体の流れを理解してからソースファイルを見ていくのね。※以前はcppを先に見て、.hは付加情報かと思っていました…orz
HeadersSyncState::ProcessingResult HeadersSyncState::ProcessNextHeaders(const
std::vector
で、ProcessingResult ProcessNextHeaders となっているのは、headerssync.hで、ProcessingResultの構造体を示していたので、このProcessingResultの型で返却して欲しい という書き方なのね。
struct ProcessingResult { std::vector<CBlockHeader> pow_validated_headers; bool success{false}; bool request_more{false}; };
Readmeを読むと以下のように書いている
A script to generate optimal parameters for the headerssync module (src/headerssync.cpp). It takes no command-line
options, as all its configuration is set at the top of the file. It runs many times faster inside PyPy. Invocation:
つまり、node間でのヘッダー情報のやり取りのことだ。c++ファイルのヘッダではない。stateで、 PRESYNC、REDOWNLOAD、FINALとある。
src/headerssync.h, src/headerssync.cpp を見てみる。
### headerssync.h
A compressed CBlockHeader, which leaves out the prevhash
HeadersSyncState
We wish to download a peer’s headers chain in a DoS-resistant way.
* – In the first download phase, called pre-synchronization, we can calculate
* the work on the chain as we go (just by checking the nBits value on each
* header, and validating the proof-of-work).
In phase 1 (presync)
In phase 2 (redownload)
#### headerssync.cpp
// The two constants below are computed using the simulation script in
// contrib/devtools/headerssync-params.py.
m_max_commitments = 6*(Ticks
downloadはこれでやってる?
if (m_current_chain_work >= m_minimum_required_work) { m_redownloaded_headers.clear(); m_redownload_buffer_last_height = m_chain_start->nHeight; m_redownload_buffer_first_prev_hash = m_chain_start->GetBlockHash(); m_redownload_buffer_last_hash = m_chain_start->GetBlockHash(); m_redownload_chain_work = m_chain_start->nChainWork; m_download_state = State::REDOWNLOAD; LogPrint(BCLog::NET, "Initial headers sync transition with peer=%d: reached sufficient work at height=%i, redownloading from height=%i\n", m_id, m_current_height, m_redownload_buffer_last_height); }
downloadのstatusに応じて処理しているのはわかるが、getHashの処理とかはheaderssync.cppの中には書いてないように見えるがどこでやってるんだろうか?
あと、基本的に、HeadersSyncState::${処理}と書かれていますね。
なんか凄いな…
mainnet=1 # mainnetを利用する txindex=1 # indexを作成して全てのトランザクションIDを参照可能にする server=1 # JSON RPCサーバとしてコマンドを受け付ける rest=1 # REST インターフェースを有効にする rpcuser= "Username" # JSON RPCのためのユーザ名 rpcpassword= "Password" # JSON RPCのためのパスワード rpcport=8332 # JSON RPC用ポート番号
gen-bitcoin-conf.sh
export LC_ALL=C TOPDIR=${TOPDIR:-$(git rev-parse --show-toplevel)} BUILDDIR=${BUILDDIR:-$TOPDIR} BINDIR=${BINDIR:-$BUILDDIR/src} BITCOIND=${BITCOIND:-$BINDIR/bitcoind} SHARE_EXAMPLES_DIR=${SHARE_EXAMPLES_DIR:-$TOPDIR/share/examples} EXAMPLE_CONF_FILE=${EXAMPLE_CONF_FILE:-$SHARE_EXAMPLES_DIR/bitcoin.conf} [ ! -x "$BITCOIND" ] && echo "$BITCOIND not found or not executable." && exit 1 DIRTY="" VERSION_OUTPUT=$($BITCOIND --version) if [[ $VERSION_OUTPUT == *"dirty"* ]]; then DIRTY="${DIRTY}${BITCOIND}\n" fi if [ -n "$DIRTY" ] then echo -e "WARNING: $BITCOIND was built from a dirty tree.\n" echo -e "To safely generate a bitcoin.conf file, please commit your changes to $BITCOIND, rebuild, then run this script again.\n" fi echo 'Generating example bitcoin.conf file in share/examples/' # create the directory, if it doesn't exist mkdir -p "${SHARE_EXAMPLES_DIR}" # create the header text cat > "${EXAMPLE_CONF_FILE}" << 'EOF' ## ## bitcoin.conf configuration file. ## Generated by contrib/devtools/gen-bitcoin-conf.sh. ## ## Lines beginning with # are comments. ## All possible configuration options are provided. To use, copy this file ## to your data directory (default or specified by -datadir), uncomment ## options you would like to change, and save the file. ## ### Options EOF # parse the output from bitcoind --help # adding newlines is a bit funky to ensure portability for BSD # see here for more details: https://stackoverflow.com/a/24575385 ${BITCOIND} --help \ | sed '1,/Print this help message and exit/d' \ | sed -E 's/^[[:space:]]{2}\-/#/' \ | sed -E 's/^[[:space:]]{7}/# /' \ | sed -E '/[=[:space:]]/!s/#.*$/&=1/' \ | awk '/^#[a-z]/{x=$0;next}{if (NF==0) print x"\n",x="";else print}' \ | sed 's,\(^[[:upper:]].*\)\:$,\ ### \1,' \ | sed 's/[[:space:]]*$//' >> "${EXAMPLE_CONF_FILE}" # create the footer text cat >> "${EXAMPLE_CONF_FILE}" << 'EOF' # [Sections] # Most options will apply to all networks. To confine an option to a specific # network, add it under the relevant section below. # # Note: If not specified under a network section, the options addnode, connect, # port, bind, rpcport, rpcbind, and wallet will only apply to mainnet. # Options for mainnet [main] # Options for testnet [test] # Options for signet [signet] # Options for regtest [regtest] EOF
sedの使い方
1.rakus rakus 2.rakus
$ sed -e ‘s/rakus/RAKUS/g’ sample.tx
1.RAKUS RAKUS
2.RAKUS
-Eのところは複数指定にしている?
${BITCOIND} --help \ | sed '1,/Print this help message and exit/d' \ | sed -E 's/^[[:space:]]{2}\-/#/' \ | sed -E 's/^[[:space:]]{7}/# /' \ | sed -E '/[=[:space:]]/!s/#.*$/&=1/' \ | awk '/^#[a-z]/{x=$0;next}{if (NF==0) print x"\n",x="";else print}' \ | sed 's,\(^[[:upper:]].*\)\:$,\ ### \1,' \ | sed 's/[[:space:]]*$//' >> "${EXAMPLE_CONF_FILE}"
シェルで引数を
#!/bin/bash if [ $# -ne 2 ]; then echo "指定された引数は$#個です。" 1>&2 echo "実行するには2個の引数が必要です。" 1>&2 exit 1 fi cat <<__EOT__ 指定された引数は、 $1 $2 の$#個です。 __EOT__ exit 0
上記をベースに、引数で表示するmanを変えていきます。caseで引数に応じて出し分ける。
#!/bin/bash output() { case $1 in "daemon") cat <<__EOT__ NAME daemon SYNOPOIS A program that resides in main memory and provides specific functions __EOT__ exit 0 ;; "cli") cat <<__EOT__ NAME cli SYNOPOIS A text-based interface that allows you to enter commands to interact with your computer's operating system __EOT__ exit 0 ;; esac } if [ $1 == "man" ]; then if [ $# -ne 2 ]; then echo "Please enter arguments after man." 1>&2 exit 1 else output $2 fi fi
$ ./test.sh man daemon
NAME daemon
SYNOPOIS A program that resides in main memory and provides specific functions
$ ./test.sh man cli
NAME cli
SYNOPOIS A text-based interface that allows you to enter commands to interact with your computer’s operating system
gen-manpage.pyだと、コンテンツは別ページに持っているようです。topディレクトリにディレクトリ名を繋げて参照しています。
os.getenvは環境変数を持ってくる
import os import subprocess import sys import tempfile BINARIES = [ 'src/bitcoind', 'src/bitcoin-cli', 'src/bitcoin-tx', 'src/bitcoin-wallet', 'src/bitcoin-util', 'src/qt/bitcoin-qt', ] # Paths to external utilities. git = os.getenv('GIT', 'git') help2man = os.getenv('HELP2MAN', 'help2man') # If not otherwise specified, get top directory from git. topdir = os.getenv('TOPDIR') if not topdir: r = subprocess.run([git, 'rev-parse', '--show-toplevel'], stdout=subprocess.PIPE, check=True, text=True) topdir = r.stdout.rstrip() # Get input and output directories. builddir = os.getenv('BUILDDIR', topdir) mandir = os.getenv('MANDIR', os.path.join(topdir, 'doc/man')) # Verify that all the required binaries are usable, and extract copyright # message in a first pass. versions = [] for relpath in BINARIES: abspath = os.path.join(builddir, relpath) try: r = subprocess.run([abspath, "--version"], stdout=subprocess.PIPE, check=True, text=True) except IOError: print(f'{abspath} not found or not an executable', file=sys.stderr) sys.exit(1) # take first line (which must contain version) verstr = r.stdout.splitlines()[0] # last word of line is the actual version e.g. v22.99.0-5c6b3d5b3508 verstr = verstr.split()[-1] assert verstr.startswith('v') # remaining lines are copyright copyright = r.stdout.split('\n')[1:] assert copyright[0].startswith('Copyright (C)') versions.append((abspath, verstr, copyright)) if any(verstr.endswith('-dirty') for (_, verstr, _) in versions): print("WARNING: Binaries were built from a dirty tree.") print('man pages generated from dirty binaries should NOT be committed.') print('To properly generate man pages, please commit your changes (or discard them), rebuild, then run this script again.') print() with tempfile.NamedTemporaryFile('w', suffix='.h2m') as footer: # Create copyright footer, and write it to a temporary include file. # Copyright is the same for all binaries, so just use the first. footer.write('[COPYRIGHT]\n') footer.write('\n'.join(versions[0][2]).strip()) footer.flush() # Call the binaries through help2man to produce a manual page for each of them. for (abspath, verstr, _) in versions: outname = os.path.join(mandir, os.path.basename(abspath) + '.1') print(f'Generating {outname}…') subprocess.run([help2man, '-N', '--version-string=' + verstr, '--include=' + footer.name, '-o', outname, abspath], check=True)
マニュアルはUbuntuだと $ man ${command} で表示される。
上部にdescriptionがあり、オプションの説明、その後はauthor, bug, report, copyrightなどだ。
以下はcatの例
CAT(1) User Commands CAT(1)
NAME
cat – concatenate files and print on the standard output
SYNOPSIS
cat [OPTION]… [FILE]…
DESCRIPTION
Concatenate FILE(s) to standard output.
With no FILE, or when FILE is -, read standard input.
-A, –show-all
EXAMPLES
cat f – g
Output f’s contents, then standard input, then g’s contents.
cat Copy standard input to standard output.
AUTHOR
Written by Torbjorn Granlund and Richard M. Stallman.
REPORTING BUGS
GNU coreutils online help:
Report any translation bugs to
COPYRIGHT
Copyright © 2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL
version 3 or later
This is free software: you are free to change and redistribute it. There is
NO WARRANTY, to the extent permitted by law.
SEE ALSO
tac(1)
Full documentation
or available locally via: info ‘(coreutils) cat invocation’
GNU coreutils 8.32 February 2022 CAT(1)
シェルで受けっとって表示か?
まず対象となるファイルを適当に作成します。
script.cpp
#include "headers.h" bool CheckSig(vector<unsigned char> vchSig, vector<unsigned char> vchPubKey, CScript scriptCode, const CTransaction& txTo, unsigned int nIn, int nHashType);
file_name = './script.cpp' with open(file_name) as f: data = f.readlines() data.insert(1, '// Copyright (c) 2009 Satoshi Nakamoto\n') with open(file_name, mode='w') as f: f.writelines(data)
これだと、全部書き変わってしまう…
copyright_header.pyだと、作者をファイルごとに定義しています。
COPYRIGHT_WITH_C = r'Copyright \(c\)' COPYRIGHT_WITHOUT_C = 'Copyright' ANY_COPYRIGHT_STYLE = '(%s|%s)' % (COPYRIGHT_WITH_C, COPYRIGHT_WITHOUT_C) YEAR = "20[0-9][0-9]" YEAR_RANGE = '(%s)(-%s)?' % (YEAR, YEAR) YEAR_LIST = '(%s)(, %s)+' % (YEAR, YEAR) ANY_YEAR_STYLE = '(%s|%s)' % (YEAR_RANGE, YEAR_LIST) ANY_COPYRIGHT_STYLE_OR_YEAR_STYLE = ("%s %s" % (ANY_COPYRIGHT_STYLE, ANY_YEAR_STYLE)) ANY_COPYRIGHT_COMPILED = re.compile(ANY_COPYRIGHT_STYLE_OR_YEAR_STYLE) def compile_copyright_regex(copyright_style, year_style, name): return re.compile(r'%s %s,? %s( +\*)?\n' % (copyright_style, year_style, name)) EXPECTED_HOLDER_NAMES = [ r"Satoshi Nakamoto", r"The Bitcoin Core developers", r"BitPay Inc\.", r"University of Illinois at Urbana-Champaign\.", r"Pieter Wuille", r"Wladimir J\. van der Laan", r"Jeff Garzik", r"Jan-Klaas Kollhof", r"ArtForz -- public domain half-a-node", r"Intel Corporation ?", r"The Zcash developers", r"Jeremy Rubin", ]
f.writelines()ではなく、f.write(”.join())で関数と引数にして書いてます。
################################################################################ # read and write to file ################################################################################ def read_file_lines(filename): with open(filename, 'r', encoding="utf8") as f: file_lines = f.readlines() return file_lines def write_file_lines(filename, file_lines): with open(filename, 'w', encoding="utf8") as f: f.write(''.join(file_lines))
なるほど、Pythonならギリまだいけるな..
様々なプログラミング言語に対し、様々なコンピュータやOSに対応させることができる共通基盤
中間言語に変換し、中間コードは、特定の機種やOSで直に動作するネイティブコードに変換して実行される
実行時に変換しながら実行するのは仮想マシン
RustやSwiftはLLVMの利用を前提としている
$ sudo apt install clang -y
$ clang -S -emit-llvm -O3 helloworld.c
$ cat helloworld.ll
; ModuleID = 'helloworld.c' source_filename = "helloworld.c" target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128" target triple = "aarch64-unknown-linux-gnu" @str = private unnamed_addr constant [14 x i8] c"Hello, world!\00", align 1 ; Function Attrs: nofree nounwind uwtable define dso_local i32 @main(i32 noundef %0, i8** nocapture noundef readnone %1) local_unnamed_addr #0 { %3 = tail call i32 @puts(i8* nonnull dereferenceable(1) getelementptr inbounds ([14 x i8], [14 x i8]* @str, i64 0, i64 0)) ret i32 0 } ; Function Attrs: nofree nounwind declare noundef i32 @puts(i8* nocapture noundef readonly) local_unnamed_addr #1 attributes #0 = { nofree nounwind uwtable "frame-pointer"="non-leaf" "min-legal-vector-width"="0" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="generic" "target-features"="+neon,+outline-atomics,+v8a" } attributes #1 = { nofree nounwind } !llvm.module.flags = !{!0, !1, !2, !3, !4, !5, !6, !7, !8} !llvm.ident = !{!9} !0 = !{i32 1, !"wchar_size", i32 4} !1 = !{i32 1, !"branch-target-enforcement", i32 0} !2 = !{i32 1, !"sign-return-address", i32 0} !3 = !{i32 1, !"sign-return-address-all", i32 0} !4 = !{i32 1, !"sign-return-address-with-bkey", i32 0} !5 = !{i32 7, !"PIC Level", i32 2} !6 = !{i32 7, !"PIE Level", i32 2} !7 = !{i32 7, !"uwtable", i32 1} !8 = !{i32 7, !"frame-pointer", i32 1} !9 = !{!"Ubuntu clang version 14.0.0-1ubuntu1.1"}
クラスの相互参照における問題
A.h
#ifndef _A_H_ #define _A_H_ #include "B.h" class A { B* m_bB; ... } #endif // _A_H_
ifndef, define, endifは重複回避
B.h
#ifndef _B_H_ #define _B_H_ #include "B.h" class B { A* m_bA; ... } #endif // _B_H_
お互いに参照していると、いつまで経っても参照が終わらなくなってしまう
#ifndef _A_H_ #define _A_H_ class B; class A { private: B* m_bB; public: A(); void foo(); void bar(); ... } #endif // _A_H_
#ifndef _B_H_ #define _B_H_ class A; class B { private: A* m_bA; public: B(A* pA); void hoge(); } #endif // _B_H_
includeはしていないのだが、うーん、なんかよくわからんな…