Heavy Lifting Done by the run time

spow workers assign mappers
assign reducers

Issues to be handled by the run time
Master data structures
– location of files created by completed mapper
– score board of mapper/reducer assignment
– fault tolerance
* start new instances if no timely resonse
* completion message from redundant stragglers
– locality management
– task granlarity
– backup tasks

Content Delivery Networks
Napster started for music sharing

content hash, node-id where content stored
content-hash = 149 => <149, 80>

Name space {key-space, node-space}
content -> sha-1 -> unique 160-bit key, unique 160-bit node id
Objective:
-> node id such that –
APIs
– put key, getkey

CDN – an overlay network
routing table at A, name, node id, next

multimedia API
middleware
commodity OS

Parallel program
-pthreads: API for parallel programs
Distributed programs
-sockets: API for distributed programs
Conventional Distributed program
client – unix(socket) – server e.g. NFS

Novel multimedia Apps
sensor-based: sensors, distributed
sense -> prioritize -> process -> actuate
computationally intensive, real time

large-scale situation awareness

PTS Programming Model
Channel ch1 =
lookup(“video channel”)
while(1){
// get data
response γ =
ch1.get()
// process data
…
// produce output
ch2.put(item,)
}