{"id":357,"date":"2016-12-06T23:53:35","date_gmt":"2016-12-06T23:53:35","guid":{"rendered":"http:\/\/redmonk.com\/fryan\/?p=357"},"modified":"2016-12-07T00:01:39","modified_gmt":"2016-12-07T00:01:39","slug":"aws-reinvent-elastic-gpus-fpgas-deep-learning-compute","status":"publish","type":"post","link":"https:\/\/redmonk.com\/fryan\/2016\/12\/06\/aws-reinvent-elastic-gpus-fpgas-deep-learning-compute\/","title":{"rendered":"AWS re:Invent &#8211; Elastic GPUs, FPGAs, Deep Learning &#038; Compute"},"content":{"rendered":"<p>Amazon Web Services annual conference, <a href=\"https:\/\/reinvent.awsevents.com\/\">re:Invent<\/a>, was held in Las Vegas last week, and to say there is a lot to unpack would be somewhat of an understatement. As my colleague <a href=\"http:\/\/redmonk.com\/sogrady\">Stephen O\u2019Grady<\/a> put it, Amazon pack more announcements into a two-day window than most companies manage in three years.<\/p>\n<p>This is the first of several posts we will be doing on various aspects of re:Invent, and in this post we will primarily focus on Deep\u00a0Learning, Elastic GPUs and FPGAs, and touch on some of the interesting compute announcements at the end.<\/p>\n<h2>Elastic GPUs<\/h2>\n<p>Earlier this year Amazon announced the <a href=\"https:\/\/aws.amazon.com\/blogs\/aws\/new-p2-instance-type-for-amazon-ec2-up-to-16-gpus\">availability of their P2 instances<\/a>, which featured up to 16GPUS for heavy machine learning. Given the comments\u00a0over the years on <a href=\"http:\/\/techblog.netflix.com\/2014\/02\/distributed-neural-networks-with-gpus.html\">GPU performance in AWS<\/a>\u00a0 it is fair to say that the P2 instances were eagerly anticipated and went a long way towards resolving many of the previous complaints around virtualized GPUs.<\/p>\n<p>However, not everyone needs even a full GPU, never mind a dedicated instance. This is what makes the Elastic GPU concept so interesting. Attach as you need it, do the work, and then move on.<\/p>\n<p>The <a href=\"https:\/\/aws.amazon.com\/blogs\/aws\/in-the-work-amazon-ec2-elastic-gpus\/\">initial Elastic GPU offering<\/a> allows you to take as little as 1\/8<sup>th<\/sup> of a GPU and use it as you need, and then move on. The offering is limited to Windows for now, with plans for support in the <a href=\"https:\/\/aws.amazon.com\/marketplace\/pp\/B01M0AXXQB\">Amazon Linux Deep Learning AMI<\/a> soon. There is one significant area of concern with the lack of <a href=\"http:\/\/www.nvidia.com\/object\/cuda_home_new.html\">CUDA support<\/a>.<\/p>\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">1\/8 GPU to a full GPU that you can attach to compute <a href=\"https:\/\/twitter.com\/hashtag\/reinvent?src=hash&amp;ref_src=twsrc%5Etfw\">#reinvent<\/a> &lt;&lt; really can&#39;t say how much this changes things for lots of users<\/p>\n<p>&mdash; Fintan Ryan (@fintanr) <a href=\"https:\/\/twitter.com\/fintanr\/status\/804000277867020288?ref_src=twsrc%5Etfw\">November 30, 2016<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>Elastic GPUs is a legitimate big deal, as are the P2 instances \u2013 and Amazons sheer volume of customers will expose GPUs to a very wide base. It is, however, worth taking note of the fact that both <a href=\"https:\/\/cloud.google.com\/gpu\/\">Google<\/a> and <a href=\"https:\/\/azure.microsoft.com\/en-gb\/blog\/azure-n-series-preview-availability\/\">Microsoft<\/a> have GPU offerings in beta as well.<\/p>\n<h2>Deep Learning Frameworks<\/h2>\n<p>As Amazon CTO Werner Vogels <a href=\"http:\/\/www.allthingsdistributed.com\/2016\/11\/mxnet-default-framework-deep-learning-aws.html\">recently highlighted<\/a>, AWS have decided to invest heavily in <a href=\"https:\/\/mxnet.io\">MXNet<\/a> as their deep learning framework of choice. \u00a0In his blog post Werner noted there key factors developers and data scientists use when selecting a deep learning framework<\/p>\n<p>&nbsp;<\/p>\n<blockquote><p>The\u00a0<strong>ability to\u00a0scale\u00a0<\/strong>to multiple GPUs (across multiple hosts) to train larger, more sophisticated models with larger, more sophisticated datasets. Deep learning models can take days or weeks to train, so even modest improvements here make a huge difference in the speed at which new models can be developed and evaluated.<\/p>\n<p><strong>Development speed\u00a0<\/strong>and programmability, especially the opportunity to use languages they are already familiar with, so that they can quickly build new models and update existing ones.<\/p>\n<p><strong>Portability\u00a0<\/strong>to run on a broad range of devices and platforms, because deep learning models have to run in many, many different places: from laptops and server farms with great networking and tons of computing power to mobiles and connected devices which are often in remote locations, with less reliable networking and considerably less computing power.<\/p><\/blockquote>\n<p>It is against these criteria that Amazon have chosen MXNet.<\/p>\n<p>What is interesting here is not just that Amazon are backing a framework, it is how explicitly they are backing a specific framework.\u00a0 Stepping back a little, there is clearly an element of <a href=\"http:\/\/www.coalitiontheory.net\/research-areas\/coalition-formation-theory\">coalition theory<\/a> emerging in the deep learning space as well. While Google and others <a href=\"http:\/\/redmonk.com\/jgovernor\/2016\/07\/15\/what-kubernetes-and-a9-tell-us-about-the-new-industry-anyone-but-amazon\/\">may be building a coalition around Kubernetes<\/a>, they have, as we have noted in the past, stolen a march on the industry with the <a href=\"http:\/\/redmonk.com\/fryan\/2016\/06\/06\/a-look-at-popular-machine-learning-frameworks\/\">general popularity of Tensorflow<\/a> \u2013 everyone else is now responding, and MXNet, regardless of its technical merits, is front and centre in this battle.<\/p>\n<div align=\"center\"><a href=\"http:\/\/redmonk.com\/fryan\/files\/2016\/12\/aws-ami-ml-fw-stars-20161206.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-medium wp-image-358\" src=\"http:\/\/redmonk.com\/fryan\/files\/2016\/12\/aws-ami-ml-fw-stars-20161206-300x300.png\" alt=\"aws-ami-ml-fw-stars-20161206\" width=\"300\" height=\"300\" srcset=\"https:\/\/redmonk.com\/fryan\/files\/2016\/12\/aws-ami-ml-fw-stars-20161206-300x300.png 300w, https:\/\/redmonk.com\/fryan\/files\/2016\/12\/aws-ami-ml-fw-stars-20161206-150x150.png 150w, https:\/\/redmonk.com\/fryan\/files\/2016\/12\/aws-ami-ml-fw-stars-20161206-768x768.png 768w, https:\/\/redmonk.com\/fryan\/files\/2016\/12\/aws-ami-ml-fw-stars-20161206-1024x1024.png 1024w, https:\/\/redmonk.com\/fryan\/files\/2016\/12\/aws-ami-ml-fw-stars-20161206-1536x1536.png 1536w, https:\/\/redmonk.com\/fryan\/files\/2016\/12\/aws-ami-ml-fw-stars-20161206-480x480.png 480w, https:\/\/redmonk.com\/fryan\/files\/2016\/12\/aws-ami-ml-fw-stars-20161206-627x627.png 627w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/div>\n<p>That said, Amazon have been very clear that they will support MXNet alongside a number of other popular deep learning frameworks such as Caffe, Torch, Theano, CNTK and of course Tensorflow.<\/p>\n<p>Additionally, it is worth taking note of the development of <a href=\"https:\/\/github.com\/amznlabs\/amazon-dsstne\/\">Amazon DSSTNE<\/a> (Deep Scalable Sparse Tensor Network Engine), and open source project from Amazon Labs. DSSTNE is focused on dealing with sparse training data \u2013 a very common problem once you step away from the headline deep learning examples of images or speech.<\/p>\n<p>Now with all of this said, we have previously noted that <a href=\"http:\/\/redmonk.com\/fryan\/2016\/06\/06\/a-look-at-popular-machine-learning-frameworks\/\">most developers will access machine and deep learning via an api<\/a>, and we still stand by that statement. At re:Invent we saw another manifestation of this approach in the announcements of <a href=\"https:\/\/aws.amazon.com\/lex\/\">Lex<\/a>, <a href=\"https:\/\/aws.amazon.com\/polly\/\">Polly<\/a> and <a href=\"https:\/\/aws.amazon.com\/rekognition\/\">Rekognition<\/a> \u2013 all of which we will return to in due course.<\/p>\n<h2>FPGAs on Demand<\/h2>\n<p>Now Amazon are well known for going their own way, and their extensive use of FPGAs has not exactly been a secret in the industry. We had the pleasure of being in the company of Amazon Distinguished Engineer James Hamilton during the analyst event and he relished in describing how Amazon are both moving repetitive tasks to silicon and controlling the associated software update cycle, aspects of which he <a href=\"https:\/\/youtu.be\/AyOAjFNPAbA\">recapped again in his keynote last Tuesday<\/a>.<\/p>\n<p>With the announce of the new F1 instances Amazon are opening the possibilities of using FPGAs to a far wider audience than ever before.<\/p>\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">FPGA dev kit, tool kits etc made available &lt;&lt; chatted about verilog several times recently, genuinely this is huge <a href=\"https:\/\/twitter.com\/hashtag\/Reinvent?src=hash&amp;ref_src=twsrc%5Etfw\">#Reinvent<\/a><\/p>\n<p>&mdash; Fintan Ryan (@fintanr) <a href=\"https:\/\/twitter.com\/fintanr\/status\/804001866535161856?ref_src=twsrc%5Etfw\">November 30, 2016<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>&nbsp;<\/p>\n<p>I had a set of conversations relatively recently with several leading financial services institutions, and one of the big reference points that I took away was a desire for, and shortage of, Verilog programmers. In any industry where speed is key, and the calculations are repetitive, offloading to FPGAs ultimately makes sense.<\/p>\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">New F1 instance &#8211; customer programmable FPGAs &lt;&lt; this is niche, but absolutely huge for specific industries <a href=\"https:\/\/twitter.com\/hashtag\/reinvent?src=hash&amp;ref_src=twsrc%5Etfw\">#reinvent<\/a><\/p>\n<p>&mdash; Fintan Ryan (@fintanr) <a href=\"https:\/\/twitter.com\/fintanr\/status\/804001167843803136?ref_src=twsrc%5Etfw\">November 30, 2016<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>This is a niche market, but it is a highly lucrative one. Right now, Amazon are set to completely own it. More importantly when you combine something like the F1 offering with the C5 Skylake instances you can begin to see some of the highest spending hedge funds, banks and trading floors moving some of their major workloads into Amazon. And with those workloads comes a massive amount of associated data which a whole host of other services can utilise.<\/p>\n<p>As a little side note, even in the world of cloud you can always be a data center geek, and the pieces <a href=\"http:\/\/perspectives.mvdirona.com\/\">James Hamilton publishes<\/a> are well worth some of your time.<\/p>\n<h2>General Compute Updates<\/h2>\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">We love ourselves some compute &#8211; <a href=\"https:\/\/twitter.com\/ajassy?ref_src=twsrc%5Etfw\">@ajassy<\/a> <a href=\"https:\/\/twitter.com\/hashtag\/reinvent?src=hash&amp;ref_src=twsrc%5Etfw\">#reinvent<\/a> <a href=\"https:\/\/t.co\/xP3LkSa5FW\">pic.twitter.com\/xP3LkSa5FW<\/a><\/p>\n<p>&mdash; Fintan Ryan (@fintanr) <a href=\"https:\/\/twitter.com\/fintanr\/status\/804001728467189760?ref_src=twsrc%5Etfw\">November 30, 2016<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>In keeping with tradition, Amazon announced a set of new compute offerings, speed &amp; feed revs and so forth. We already touched on the new C5 instances (think highly optimized compute tasks, machine learning), the other two worth explicitly calling out are the R4 instances which provide up to 488 GiBs of memory (think in memory databases, realtime analytics, Spark clusters and so forth) and the I3 instances which massive IO improvements (think high performance database workloads).<\/p>\n<p><strong>Credits<\/strong>: JTAG image by <a href=\"https:\/\/www.flickr.com\/photos\/amagill\/2877921124\">Andrew Magill on Flickr<\/a>, <a href=\"https:\/\/creativecommons.org\/licenses\/by\/2.0\/\">CC2.0 license<\/a>.<\/p>\n<p><strong>Disclaimers<\/strong>: Amazon paid my T&amp;E for re:Invent. Amazon and Microsoft are current RedMonk clients.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Amazon Web Services annual conference, re:Invent, was held in Las Vegas last week, and to say there is a lot to unpack would be somewhat of an understatement. As my colleague Stephen O\u2019Grady put it, Amazon pack more announcements into a two-day window than most companies manage in three years. This is the first of<\/p>\n","protected":false},"author":40,"featured_media":359,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[38,4,20,27,31,7,37,15,32,17,28],"tags":[],"class_list":["post-357","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-aws","category-business","category-cio","category-cloud","category-cto","category-data","category-deep-learning","category-developers","category-frameworks","category-infrastructure","category-machine-learning"],"acf":[],"jetpack_featured_media_url":"https:\/\/redmonk.com\/fryan\/files\/2016\/12\/2877921124_d09ac8cf96_z.jpg","_links":{"self":[{"href":"https:\/\/redmonk.com\/fryan\/wp-json\/wp\/v2\/posts\/357","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/redmonk.com\/fryan\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/redmonk.com\/fryan\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/redmonk.com\/fryan\/wp-json\/wp\/v2\/users\/40"}],"replies":[{"embeddable":true,"href":"https:\/\/redmonk.com\/fryan\/wp-json\/wp\/v2\/comments?post=357"}],"version-history":[{"count":0,"href":"https:\/\/redmonk.com\/fryan\/wp-json\/wp\/v2\/posts\/357\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/redmonk.com\/fryan\/wp-json\/wp\/v2\/media\/359"}],"wp:attachment":[{"href":"https:\/\/redmonk.com\/fryan\/wp-json\/wp\/v2\/media?parent=357"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/redmonk.com\/fryan\/wp-json\/wp\/v2\/categories?post=357"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/redmonk.com\/fryan\/wp-json\/wp\/v2\/tags?post=357"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}