HPC and the Cloud Ryan Enge Compute Canada/WestGrid/UVic Systems Administrator University of Victoria Beta Testing Only! Overview * 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
What is OpenStack OpenStack Infrastructure OpenStack Components Accessing OpenStack OpenStack Horizon OpenStack CLI OpneStack Instances (examples) HPC Benefits HPC Example Use Cases HPC Instance Demos Getting data in/out of VM’s What is OpenStack? “OpenStack is a free and open-­‐source software cloud computing software platform.[2] Users primarily deploy it as an infrastructure as a service (IaaS) solution. The technology consists of a series of interrelated projects that control pools of processing, storage, and networking resources throughout a data center—which users manage through a web-­‐based dashboard, command-­‐line tools, or a RESTful API. OpenStack.org release it under the terms of the Apache License.” (Wikipedia) OpenStack Infrastructure OpenStack (http://www.openstack.org) *  3 Control Nodes in HA Active/Active LB Configuration *  2 Neutron Nodes in HA Active/Passive *  40 Compute nodes for VM instances Ceph (http://ceph.com) *  3 Monitor nodes *  13 OSD servers (12x4TB drives/server) *  1 RADOS Gateway (Swift api/Object storage) *  ~600TB RAW/~200TB Useable (3 replicas) *  No automated backups at this time *  No POSIX access at this time OpenStack Infrastructure System Type Cores CPU Memory Ratio Interconnect Storage Persistent 32(640) 10:1* 256GB/node 10 GigE Ephemeral 480(960) 1:1 256 GB/
node 10 GigE ~1.6TB local/200 TB Ceph Ephemeral Big Mem 128(256) 1:1 512 GB/node 10 GigE ~1.6TB local/200 TB Ceph ~200TB Ceph OpenStack Components * 
* 
* 
* 
* 
* 
* 
* 
* 
Nova (compute) Keystone (identity service) Horizon (dashboard) Glance (image service) Neutron (networking) Cinder (block storage) Swift (object storage) Ceilometer (accounting service) Ironic* (bare metal provisioning) OpenStack Component Overview Mirantis www.mirantis.com Accessing OpenStack *  Web Browser (https://nefos.westgrid.ca) this URL will change https://west.cloud.computecanada.ca *  Command Line (python clients: nova, neutron, etc, new openstack client) Horizon Demo OpenStack Horizon Demo CLI Interface Demo Windows VM Example HPC Benefits * 
* 
* 
* 
* 
* 
* 
* 
Flexibility Persistent VM’s for Non-­‐traditional HPC uses Full control of OS and software (limitless possibilities) Large memory available for jobs Software Defined Networks (SDN) Create virtual clusters Cloud scheduler (provides batch processing in the cloud) Cloud bursting HPC Example Use Cases *  Single VM’s *  Windows OS *  Interactive VM’s (ipython notebook, R shiny) *  Persistent VM’s (http://206.12.48.70:3838/examples/
01_hello/) *  Multiple VM’s *  Virtual Cluster (torque/moab) *  MPI *  Hadoop Demo HPC Instance Demos Getting Data in/out *  Owncloud (client, cadaver, davfs2) *  Globus On-­‐line *  Sshfs *  Scp *  Rsync Thank You! Questions? 

Applications

Services aux utilisateurs

isda2014_christian_ohlwein - International Symposium on Data