Zap

of 39

Please download to get full document.

View again

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
PDF
39 pages
0 downs
0 views
Share
Description
A System for Migrating Computing Environments. Zap. Steven Osman Dinesh Subhraveti Gong Su Jason Nieh. Benefits of Migration. Dynamic Load Balancing Mobility Data Access Locality Improved Administration Fault Resilience. Clustered System Approach. Single system image across a cluster
Transcript
A System for Migrating Computing EnvironmentsZapSteven OsmanDinesh SubhravetiGong SuJason NiehBenefits of Migration
  • Dynamic Load Balancing
  • Mobility
  • Data Access Locality
  • Improved Administration
  • Fault Resilience
  • Clustered System Approach
  • Single system image across a cluster
  • Good for load-balancing
  • Examples include, MOSIX, Sprite
  • May leave dependency on previous host
  • May be new operating system or significant kernel changes
  • Middleware/Language Approach
  • Object-based approach using special programming language or middleware
  • Examples include, Abacus, Emerald, Globus, Legion, Rover
  • Requires applications to be rewritten
  • User-level Approach
  • No operating system changes
  • Good for long-running applications
  • Examples include, Condor, CoCheck, Libckpt, MPVM
  • Does not support many common operating system services
  • Virtual Machine Monitor Approach
  • Support any operating system
  • No application changes
  • Example, using VMware for migration
  • Must migrate the whole operating system
  • Potentially higher overhead
  • Introducing Zap
  • Transparent migration
  • Unmodified legacy applications
  • Networked applications
  • Commodity operating system
  • Minimal operating system changes
  • Leaves nothing behind
  • Low overhead
  • Outline
  • Background & Motivation
  • Difficulties of Migration
  • Zap components
  • Virtualization
  • Migration
  • Experimental Results
  • Conclusion
  • Migration Difficultiesint iChildPID;if (iChildPID=fork()) { /* parent does some work */ waitpid(iChildPID);} else { /* child does some work */ exit(0);}PID 10PID 30PID 10PID 20PID 40PID 20Resource Consistency ProblemHost AHost BParent invoked waitpid(20) PID 10PID 10PID 10PID 20PID 20PID 20Resource Conflict ProblemHost AHost BPID 20PID 20Resources May Conflict With Other ProcessesParentResource Dependency ProblemHost AHost BParentParentChildParent and child depend on each otherProblem RecapResource consistency
  • Names can’t change
  • Resource conflict
  • Names can’t be duplicates
  • Resource dependency
  • Migration must be complete
  • Solution
  • Group processes into a POD (Process Domain) that has a private virtual namespace
  • PODs can contain one process, one group of processes, or a whole user session
  • PODs are migrated as a unit
  • Solves
  • Resource consistency problem
  • Resource conflict problem
  • Resource dependency problem
  • Zap ArchitectureZap combines
  • Thin virtualization layer
  • Checkpoint/restart mechanism
  • Checkpoint/restart offers:
  • Easier to implement than demand paging
  • Leaves nothing behind
  • Suspend sessions
  • Easily configure and clone environments
  • Dynamic system configuration
  • What Should Zap Virtualize?
  • Process identifiers (PIDs)
  • Inter-process communication (IPC) keys
  • Memory
  • File system structure
  • Network connections
  • Devices
  • PID and IPC Key Virtualization & Migration
  • Create unique namespace for the POD
  • Names are virtualized
  • When entering a system call, replace POD virtual identifiers with real ones
  • When exiting a system call, replace real return values with POD virtual ones
  • Mask out identifiers that do not belong to the POD
  • Memory Virtualization & Migration
  • Like IPC, create unique shared memory namespace
  • Modern architectures support virtual memory
  • Thank you modern architectures!Migration optimization: Move only data pages, code pages can be remappedFile System Virtualization & Migration
  • Some filenames can’t conflict:
  • /var/run/httpd.pid
  • Some directories have unique configuration:
  • /etc
  • Some directories depend on the current processes
  • /procFile System Virtualization & Migration
  • Create a directory structure for POD
  • Use network file systems
  • Create private POD directories
  • Good for /tmp, /var & POD specific configuration
  • Private /proc directory
  • Private /dev directory
  • File System ExampleHost FSbinetcpod bin  NFS:/pods/bin dev  Dynamic proc Dynamic tmp  Private PODPOD FSUse chroot() to map POD root directoryNetworking Virtualization & Migration
  • Two network addresses:
  • Persistent internal address
  • Host-dependent external address
  • For connection migration:
  • Transport layer sees virtual address
  • Network layer sees real address
  • Transport layer independent
  • Initial virtual address is real address
  • 12=32Application12=3212=12Application21=2121=2121=2332121212NetworkADDR. 1NetworkADDR. 2NetworkADDR. 3NetworkADDR. 2NetworkADDR. 1NetworkADDR. 2Virtual NetworkingTransportADDR. 1TransportADDR. 2Device Virtualization & MigrationDevice migration is hard
  • Pseudo Terminal
  • Sound Device
  • CDRW During a Recording Session
  • Electron Microscope
  • Device Migration & VirtualizationPseudo Terminal  Virtual device configuration+dataSound Device  Virtual device configurationRecording CDRW  Migrate laterElectron Microscope  Communicate with original hostDevice Migration & VirtualizationUnsupported devices do not appear in a POD’s /devZap currently supports pseudo terminals, ensuring their names are consistent after migration (e.g. /dev/pts/2)Zap Implementation
  • Developed for Linux 2.4
  • Zap design enables
  • Loadable kernel module
  • No need to rebuild the kernel
  • Intercept system calls for virtualization
  • Zap ImplementationUser spaceUser Processeskernel spaceZAP VirtualizationSystem CallsZap MigrationKernelVirtualization Cost
  • Created micro-benchmarks
  • PID calls (getpid)
  • IPC calls (shmget/ctl, semget/ctl)
  • Process creation calls (fork, execve, exit)
  • Used macro-benchmarks
  • Apache
  • Build Linux kernel
  • Volano
  • Virtualization ResultsVirtualization Results
  • Zap incurs low overhead
  • Migration Cost – VNC SessionMigration Cost – Apache
  • Apache 2.0.35
  • Default configuration
  • Migration Cost – TimeMigration Cost – SpaceMigration Cost
  • Zap can be fast
  • <1 second checkpoint/restart times
  • Includes Zap networking round-trip
  • Zap
  • Offers transparent migration of legacy and network applications
  • Introduces PODs
  • Consistency
  • Conflict free
  • Avoids Unwanted dependencies
  • Leaves nothing behind
  • Fast and lightweight
  • For more information…
  • Zap computing
  • http://www.ncl.cs.columbia.edu/research/migrate
  • Network Computing Laboratory
  • http://www.ncl.cs.columbia.edu/Future Work
  • Secure migration
  • Trusted images, POD sandbox, etc.
  • Generalized device support
  • Migration policies
  • Heterogeneity
  • Contextualization
  • Resource management
  • We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks