ソフトウェアエンジニアの技術ブログ：Software engineer tech blog

Features
– parts of an image that encode it in a compact form

Edges
edges in an image
surface normal, depth, surface color, illumination
information theory view: edges encode change, therefore edges efficiently encode an image

Edges appear as ridges in the 3D height map of an image

Edge Detection
Basic Idea
Look for a neighborhood with strong signs of change
Derivatives of F(x,y) to get Edges

Need an operation that when applied to an image returns its derivatives
model these “operatiors” as mask/kernel
then “threshold” this gradient function to select edge pixels

Gradient of an image = measure of change in Image function.
ΔF = [δF/δx,δF/δy]
measure of change in image function (F) in x and y
Gradient direction is
Θ = tan^-1[δF/δx,δF/δy]

1.cross-correlation
2.convolution
3.difference between cross-correlation and convolution
4.properties of these methods!

A mathematical representation for smoothing
F[3,3]
F[i,j]

In signal processing, cross-correlation is a measure of similarity of two waveforms as a function of a time-lag applied to one of them.
Also known as a sliding dot product or sliding inner-product

Cross-Correlation method
G[i,j] = kΣu=-k*kΣv=-k*h[u,v]F[i+u,j+v]

Box Filter
Gaussian Filter
size:21×21
Values: Gaussian or Normal distribution
σ1, 3, 6, 9
F[i,j], h[i,j], G[i,j]
G[i,j] = kΣu=-k*kΣv=-k*h[u,v]F[i-u,j-v]
Denoted by G = h * F
Flip filter in both dimensions
bottom to top
right to left
Then apply cross-correlation

Linear and shift invariants
behaves the same everywhere
the value of the output depends on the pattern in the image neighborhood, not the position of the neighborhood

identity: unit umpulse
E = […0,0,1,0,0…]
F * E = F
true of cross-correlation

Separable
if the filter is separable, convolve all rows, then convolve

0, 0, 0
0, 1, 0
0, 0, 0
x 2 =
1/9, 1/9, 1/9
1/9, 1/9, 1/9
1/9, 1/9, 1/9

1.Smooth an image over a neighborhood of pixels
2.Median filtering as a special non-linear filtering and smoothing approach

Determine the average of neighboring values
1. moving average [1 1 1 1 1] x 1/5
2. weighted moving average [1 4 6 4 1] x 1/16

smoothing process over an image using averages

some observation
-small image “rubbed” over bigger image
kernel h(i,j)
3×3 area around each original pixel used (i.e., neighborhood size k=1)
window size is 2k+1, therefore our wind is 3×3
G[i,j] = 1/(2k+1)^2 ΣΣF[i+u,j+v]

Special Case: Median Filtering
F[i,j]
median and average is changed value
Apply median filter to all 3*3 regions over the whole image
Median filtering: nonlinear operation often used in image processing

reduces noise, but
preserves edges(sharp lines!)
main idea: use median of all pixel in area, instead of mean

Median Filtering for noise removal

-Divide (brightens photos)
-Addition (too many whites)
-Subtract (too many blacks)
-Difference (subtract with scaling)
-Darken fblend(a,b)=min(a,b) for RGB
-Lighten fblend(a,b)=max(a,b) for RGB

multiply fblend(a,b)=ab
darker
Screen fblend(a,b)=1-(1-a)(1-b)
brighter

Dodge and burn
-dodge and burn change the lightness of the pictures
-dodging lightens an image, while burning darkens it
-dodge builds on screen mode
-burn build on multiply mode
-there are numerous variations of each!

Darken

-a real life example of point arithmetic
-variety of blending modes build on concept of point processes

a bit about the setup
the final output is a pixel blend
-> screen capture + camera

Blending Pixels
-blend two pixels from two images: fblend(a,b)
For example
-average: fblend(a,b)=(a+b)/2 : lighter color
-normal: fblend(a,b)=b

-order of color channels
-image compression information
-matadata about pictures
(Exchangeable image file format EXIF, etc.)

point-process computations
add/subtract images
a-blending & its applications
image histograms

i(x,y)

How to obtain discrete values?
– sample the two-dimensional (2D) space on a regular grid
– quantize each sample (round to “nearest integer”)
Result: matrix of integer values (range, e.g. 0-255)

Point-Process: Pixel/Point Arithmetic
subtract value, if it is negative, it would be Black
image 1 – image 2
binary(image 1 – image 2)

CD+AE+LD
-> whole are become over 255

alpha-blending
.34 CD + .34 AE + .34 LD
transparency(conversely, opacity)
usually represented as α
α　varies from 0 to 1.0
RGB -> αRGB 0-1, 0-255

PImage img;

void setup(){
	size(345, 567);
	// The image file must be in the data  folder of the current sketch
	// to load successfully
	img = loadImage("buzz3.jpg"); // Load the image into the program
}

void draw(){
	// display the image at its actual size at point(0,0)
	image(img, 0, 0);
	// Displays the image at point(0, height/2) at half of its size
	// image(img, 0, height/2, img.width/2, img.height/2);
}

openCV
here you can download
http://opencv.org/releases.html
http://opencv.jp/

in python-openCV

import cv2
img = cv2.imread('input.png')
cv2.imwrite('output.png', img)

img = imread('input.png')
imwrite('output.png', img)

imshow(img)
imageinfo('input.png')
help images

Typically, the functional operation requires discrete values
-sample the two-dimensional(2D) space on a regular grid
-Quantize each sample(rounded to “nearest integer”)
Matrix of integer values(Range:0-255)

image, height map, zoomed in
Pixel count
Image Histogram, intensity bins

color, red channel, green channel, blue channel

Raster image formats store a series of colored dot “pixels”
Number of bits for each pixel represents the depth of color
– 1 bit-per-pixel: 2colors (black or white, binary)
– 4 bit-per-pixel: 16colors
– 8 bit-per-pixel: 256 different colors 2^8 = 256

24 bit per pixel usually means 8 bits per color
at the two highest levels, the pixels themselves can carry up to 16777,216 different colors

Common raster image format
-gif, jpg, ppm, tif, bmp, etc

Exercise to do on own
-openCV/Python
(opencv.org, python.prg)
-matlab/octave
(mathworks.com, www.gnu.org/software/octave)
-Processing
(processing.org)

import cv2

# Read images
img = cv2.imread("mona-lisa.png")
print "Read image from file; size: {}x{}".format(img.shape[1], img.shape[0])
cv2.imshow("Mona Lisa", img)

# and run different operations on them
img_smooth = cv2.GaussianBlur(img, (7, 7), 0) #smooth image using a Gaussian Blur
cv2.imshow("Smoothed", img_smooth)

img_gray = cv2.cvtColor(img_smooth, cv2.COLOR_BGR2GRAY)
cv2.imshow("Grayscale", img_gray)

buzz=imread('buzz3.jpg')
imshow(buzz) 

A “picture element” that contains the light intensity at some location (i,j) in the image
i, j
I(i,j) = some numeric value

width * height
= 1280 * (1280 /2)
= 0.8192mp image

Characteristics of a digital image
Original Image, Zoomed In, Values
Piots of values at a slice

A two-dimensional array of pixels and respective intensities
image can be represented as a matrix
intensity value range from 0 = black to 255 = white
two dimension array

8 bits allow to represent 2^8 = 256 different values

Digital image is a Function
x or i, y or j

Rays of light -> Picture
Illumination, optics, sensor, processing, display

make an image a “computable” object
A digital image

digital image – pixel and image resolution
discrete (matrix) and continuous (function) representations
grayscale and color images
digital image formats

A digital image(W X H)
column and rows
width, height
square image -> 512 x 512 pixels = 262 = 0.26mp image
mp: mega pixel image

Numeric representation in 2-D (x and y)
Referred to as I(x,y) in continuous function from I(i,j) in discrete
Image resolution expressed in terms of width and height of the image