CSE 124 Lecture Notes - Lecture 21: Information Retrieval
22 May 2018
School
Department
Course
Professor
Notes on project 2
- NO need to handle “slashes” in filename!
- Client needs to check BlockStore to see if ANY of the blocks already exist
- you should not NEEDLESSLY upload them again
- only send blocks that are different
[client] config upload /tmp/foo.txt
[client] config download foo.txt /var/tmp
- analyze the files in the directory “tmp” and see if blocks ALREADY exist
- config will “index” into some local state
Hosting: Multiple Machines per Site
Problem: Overloaded popular web site
- Replicate the site across multiple machines to handle the load
- Want to direct client to a “particular” replica. WHY?
- balance load across server replicas
Solution #1: manual selection by clients
- Each replica has its OWN site name
Hosting: Load-Balancer Approach
Solution #2: SINGLE IP address, multiple machines
- “Run” multiple machines behind a single IP address
-Load-balancer: network device w/ a single-to-multiple interface
- Accepts traffic
- Inspects and farms traffic to different replicas based on an algorithm
-
- Ensures ALL packages from a single TCP connection go to the same replica
- We can “Round-robin” connections to different machines, but can we “round-robin”
packets? NO. Servers could be receiving packets it has NO idea how to deal with them
Solution #3: DNS Redirection Approach
- multiple IP addresses, multiple machines
- same DNS, but “different” IP for each replica
- DNS returns addresses “round-robin”
Factors of Variable Response Time
- these approaches do not ALWAYS perform “equally”
Reasons for variation:
- Shared Resources (local) - CPU cores, processor caches, bandwidth
- Global Resource Sharing - shared file systems, network switches
- Daemons - scheduled procedures
Maintenance Activities
Document Summary
No need to handle slashes in filename! Client needs to check blockstore to see if any of the blocks already exist. You should not needlessly upload them again. Analyze the files in the directory tmp and see if blocks already exist. Config will index into some local state. Replicate the site across multiple machines to handle the load. Want to direct client to a particular replica. Each replica has its own site name. Run multiple machines behind a single ip address. Inspects and farms traffic to different replicas based on an algorithm. Ensures all packages from a single tcp connection go to the same replica. Servers could be receiving packets it has no idea how to deal with them. Same dns, but different ip for each replica. These approaches do not always perform equally . Shared resources (local) - cpu cores, processor caches, bandwidth. Global resource sharing - shared file systems, network switches.
