{"id":349,"date":"2007-03-01T22:44:51","date_gmt":"2007-03-02T05:44:51","guid":{"rendered":"https:\/\/staging.opexlearning.com\/resources\/349\/youtubes-queueing-properties"},"modified":"2014-10-03T15:48:04","modified_gmt":"2014-10-03T20:48:04","slug":"youtubes-queueing-properties","status":"publish","type":"post","link":"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/","title":{"rendered":"Youtube Upload Processing and Queueing Theory"},"content":{"rendered":"<div class=\"a296a24fa2fc69ef5487857f02f111e8\" data-index=\"9\" style=\"float: none; margin:10px 0 10px 0; text-align:center;\">\n<script async src=\"\/\/pagead2.googlesyndication.com\/pagead\/js\/adsbygoogle.js\"><\/script>\r\n<!-- Single Post readerboard -->\r\n<ins class=\"adsbygoogle\"\r\n     style=\"display:inline-block;width:728px;height:90px\"\r\n     data-ad-client=\"ca-pub-8207522353004717\"\r\n     data-ad-slot=\"1144967431\"><\/ins>\r\n<script>\r\n(adsbygoogle = window.adsbygoogle || []).push({});\r\n<\/script>\n<\/div>\n<p>YouTube Upload Processing has many intriguing queueing properties.\u00a0 This article will primarily look at the mpeg-to-swf conversion and study out the queueing properties of that process.\u00a0You can also view all 40+ articles on <a title=\"articles on queueing theory\" href=\"https:\/\/staging.opexlearning.com\/resources\/queueing-theory\/\">Queueing Theory<\/a>.<\/p>\n<p>I&#8217;ll show the basic process of how to upload a video on Youtube, explain the Queueing mechanics that goes on behind-the-scenes, propose a few books in case you&#8217;re interested in Queueing Theory applications, and then finally show a video tutorial on an introduction to Queueing Theory and Models.<\/p>\n<p>I make many assumptions in this article, so feel free to poke holes in what I&#8217;m arguing.<\/p>\n<p><strong>Assumptions and Data<\/strong><\/p>\n<p>Below is a very simple, high-level process map of the steps to upload a video on YouTube:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-15163\" src=\"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2007\/03\/youtube-upload-queueing-process.png\" alt=\"upload youtube videos, queueing process\" width=\"722\" height=\"111\" srcset=\"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2007\/03\/youtube-upload-queueing-process.png 722w, https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2007\/03\/youtube-upload-queueing-process-600x92.png 600w, https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2007\/03\/youtube-upload-queueing-process-300x46.png 300w\" sizes=\"(max-width: 722px) 100vw, 722px\" \/><\/p>\n<div><\/div>\n<div>In summary, the user does the following:<\/div>\n<ol>\n<li>User uploads video<\/li>\n<li>User waits while video is uploaded<\/li>\n<li>Upload is completed<\/li>\n<li>User waits for video to be converted from MPEG to SWF<\/li>\n<li>Conversion from MPEG to SWF completes<\/li>\n<\/ol>\n<p>Based on a small sample of 20 uploaded videos of an average file size of 3.5 MB, I calculate the mean for uploads to be ~180 seconds per MB and ~240 seconds per MB for the conversion.<\/p>\n<p>Based on YouTube&#8217;s own disclosure, we also know that there are on average 65,000 video uploads per day.\u00a0 We do not know the average file size.<\/p>\n<p><strong> Queueing Properties<\/strong><\/p>\n<p>Important items to note when studying the queueing properties of a system are the following:<\/p>\n<ol>\n<li>\u00ce\u00bb = Arrival Rate, or more specific, the time between arrivals.\u00a0 For most queues, we can assume that the arrival distribution can be approximated by a Poisson distribution; which means that the time between arrivals are not deterministic, but random.<\/li>\n<li>\u00ce\u00bc = Service Rate, or more specific the time for a arrival to be serviced.<\/li>\n<\/ol>\n<p>A poisson distribution typically looks skewed to the left or to the right &#8212; that is because the mean and the standard deviation is the same.\u00a0 Here&#8217;s a standard picture of a poisson for server utilization:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-15164\" src=\"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2007\/03\/inter-arrival-rate-chart.gif\" alt=\"chart of inter-arrival rates in a queue\" width=\"294\" height=\"218\" \/><\/p>\n<div><\/div>\n<p>What we see above is that as there are more simultaneous connections, there is a subsequent arrival rate batching &#8212; represented by the poisson curve above.<\/p>\n<p>Given the data and notation, we can now attempt to better understand the queueing properties of the MPEG-to-SWF conversion.\u00a0 <strong>Remember<\/strong>: the data I have assumes several things and, is most likely, completely off the mark.\u00a0 But, it&#8217;s an attempt and, if anything, it&#8217;s fun to try.<\/p>\n<p><strong>Presuppostions Redux<\/strong><\/p>\n<p>So, 65,000 video uploads on a 12 hour day, gives us the following:<\/p>\n<p>\u00ce\u00bb = 65,000 \/ 720 minutes = <strong>(90 \/ minute)<br \/>\n<\/strong>\u00ce\u00bc = 3.5 * 240 seconds = 840 seconds; 840 seconds \/ 60 = <strong>(14 conversion \/ minute)<\/strong><\/p>\n<p>Arguably, 14 conversion \/ minute is very low.\u00a0 Let&#8217;s just assume that YouTube average service rate is <strong>200 conversions \/ minute<\/strong>.\u00a0\u00a0 Given that, we can now learn about the queueing properties, which I describe below.<br \/>\n<strong><br \/>\nAverage Number of Videos Waiting to be Converted<\/strong><\/p>\n<p>The equation to learn about the average number of files in the conversion process is the following:<\/p>\n<blockquote><p>C<sub>w<\/sub> = (\u00ce\u00bb<sup>2<\/sup> \/ \u00ce\u00bc(\u00ce\u00bc &#8211; \u00ce\u00bb))<\/p><\/blockquote>\n<p>So,<\/p>\n<blockquote><p>C<sub>w = <\/sub>[(8100) \/ (200(200 &#8211; 90)] = <strong>.36<\/strong><\/p><\/blockquote>\n<p>So, given the assumptions above, not even 1 video is waiting to be converted from MPEG-to-SWF.<\/p>\n<p><strong>Average Number of Videos in the Conversion Process<\/strong><\/p>\n<blockquote><p>C<sub>s<\/sub> = (\u00ce\u00bc &#8211; \u00ce\u00bb)<\/p><\/blockquote>\n<p>So,<\/p>\n<blockquote><p>C<sub>s = <\/sub>[(90) \/ (200 &#8211; 90)] = <strong>.81<\/strong><\/p><\/blockquote>\n<p>This means, given the assumptions above, that an any point in time, there is 1 video in the conversion system.<\/p>\n<p><strong>Average Time Spent Waiting<\/strong><\/p>\n<blockquote><p>T<sub>w<\/sub> = (\u00ce\u00bb \/ \u00ce\u00bc(\u00ce\u00bc &#8211; \u00ce\u00bb))<\/p><\/blockquote>\n<p>So, we get:<\/p>\n<blockquote><p>T<sub>w = <\/sub>[(90) \/ (200(200 &#8211; 90))] = <strong>.004<\/strong><\/p><\/blockquote>\n<p>This means as videos enter the conversion process, there is hardly any waiting &#8212; they are served almost immediately.<\/p>\n<p><strong>Average Time Spent in the System<\/strong><\/p>\n<blockquote><p>T<sub>s<\/sub> = (1 \/ (\u00ce\u00bc &#8211; \u00ce\u00bb))<\/p><\/blockquote>\n<p>So, we get,<\/p>\n<blockquote><p>T<sub>s = <\/sub>(1 \/ (200 &#8211; 90)) = <strong>.009<\/strong><\/p><\/blockquote>\n<p>This means, then, that as videos are uploaded and enter the conversion queue, they are served almost immediately, without any waiting.<\/p>\n<p><strong>Weaknesses in Analysis<\/strong><\/p>\n<p>Okay, the numbers above are pretty much pulled out of the clouds.\u00a0 But, if we had real data, then you could just plug them into the equations above.\u00a0 My guess is, though, that even if we had real numbers, the results would be really close to what I show above.\u00a0 Why?\u00a0 Well, for each input into the YouTube system, one could argue that it has very little impact on resources &#8212; this is a common property in telephony and in server modeling.\u00a0 I see the same thing going on with YouTube.\u00a0 The biggest challenge for YouTube is not computing resources, but storage capacity.<\/p>\n<!--CusAds0-->\n<div style=\"font-size: 0px; height: 0px; line-height: 0px; margin: 0; padding: 0; clear: both;\"><\/div>","protected":false},"excerpt":{"rendered":"<p>YouTube Upload Processing has many intriguing queueing properties.\u00a0 This article will primarily look at the mpeg-to-swf conversion and study out the queueing properties of that process.\u00a0You can also view all 40+ articles on Queueing Theory. I&#8217;ll show the basic process of how to upload a video on Youtube, explain the Queueing mechanics that goes on [&hellip;]<\/p>\n","protected":false},"author":12327,"featured_media":9278,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[29],"tags":[641,643,642,640],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.0 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Youtube Upload Processing and Queueing Theory<\/title>\n<meta name=\"description\" content=\"article shows the application of queueing theory to youtube upload processing\" \/>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Youtube Upload Processing and Queueing Theory\" \/>\n<meta property=\"og:description\" content=\"article shows the application of queueing theory to youtube upload processing\" \/>\n<meta property=\"og:url\" content=\"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/\" \/>\n<meta property=\"og:site_name\" content=\"OpEx Learning\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/get.shmula\" \/>\n<meta property=\"article:published_time\" content=\"2007-03-02T05:44:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2014-10-03T20:48:04+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2011\/09\/queueing-theory-application-business.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"437\" \/>\n\t<meta property=\"og:image:height\" content=\"317\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Uday Kawar\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@shmula\" \/>\n<meta name=\"twitter:site\" content=\"@shmula\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Uday Kawar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/\",\"url\":\"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/\",\"name\":\"Youtube Upload Processing and Queueing Theory\",\"isPartOf\":{\"@id\":\"https:\/\/staging.opexlearning.com\/resources\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2011\/09\/queueing-theory-application-business.jpg\",\"datePublished\":\"2007-03-02T05:44:51+00:00\",\"dateModified\":\"2014-10-03T20:48:04+00:00\",\"author\":{\"@id\":\"https:\/\/staging.opexlearning.com\/resources\/#\/schema\/person\/9335b5223b67189b35bda7d6be11c3fd\"},\"description\":\"article shows the application of queueing theory to youtube upload processing\",\"breadcrumb\":{\"@id\":\"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/#primaryimage\",\"url\":\"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2011\/09\/queueing-theory-application-business.jpg\",\"contentUrl\":\"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2011\/09\/queueing-theory-application-business.jpg\",\"width\":437,\"height\":317,\"caption\":\"business applications for queueing theory\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/staging.opexlearning.com\/resources\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Queueing Theory\",\"item\":\"https:\/\/staging.opexlearning.com\/resources\/queueing-theory-posts\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Youtube Upload Processing and Queueing Theory\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/staging.opexlearning.com\/resources\/#website\",\"url\":\"https:\/\/staging.opexlearning.com\/resources\/\",\"name\":\"OpEx Learning\",\"description\":\"Lean Six Sigma. Simplified.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/staging.opexlearning.com\/resources\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/staging.opexlearning.com\/resources\/#\/schema\/person\/9335b5223b67189b35bda7d6be11c3fd\",\"name\":\"Uday Kawar\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Youtube Upload Processing and Queueing Theory","description":"article shows the application of queueing theory to youtube upload processing","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Youtube Upload Processing and Queueing Theory","og_description":"article shows the application of queueing theory to youtube upload processing","og_url":"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/","og_site_name":"OpEx Learning","article_publisher":"https:\/\/www.facebook.com\/get.shmula","article_published_time":"2007-03-02T05:44:51+00:00","article_modified_time":"2014-10-03T20:48:04+00:00","og_image":[{"width":437,"height":317,"url":"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2011\/09\/queueing-theory-application-business.jpg","type":"image\/jpeg"}],"author":"Uday Kawar","twitter_card":"summary_large_image","twitter_creator":"@shmula","twitter_site":"@shmula","twitter_misc":{"Written by":"Uday Kawar","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/","url":"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/","name":"Youtube Upload Processing and Queueing Theory","isPartOf":{"@id":"https:\/\/staging.opexlearning.com\/resources\/#website"},"primaryImageOfPage":{"@id":"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/#primaryimage"},"image":{"@id":"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/#primaryimage"},"thumbnailUrl":"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2011\/09\/queueing-theory-application-business.jpg","datePublished":"2007-03-02T05:44:51+00:00","dateModified":"2014-10-03T20:48:04+00:00","author":{"@id":"https:\/\/staging.opexlearning.com\/resources\/#\/schema\/person\/9335b5223b67189b35bda7d6be11c3fd"},"description":"article shows the application of queueing theory to youtube upload processing","breadcrumb":{"@id":"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/#primaryimage","url":"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2011\/09\/queueing-theory-application-business.jpg","contentUrl":"https:\/\/staging.opexlearning.com\/resources\/wp-content\/uploads\/2011\/09\/queueing-theory-application-business.jpg","width":437,"height":317,"caption":"business applications for queueing theory"},{"@type":"BreadcrumbList","@id":"https:\/\/staging.opexlearning.com\/resources\/youtubes-queueing-properties\/349\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/staging.opexlearning.com\/resources\/"},{"@type":"ListItem","position":2,"name":"Queueing Theory","item":"https:\/\/staging.opexlearning.com\/resources\/queueing-theory-posts\/"},{"@type":"ListItem","position":3,"name":"Youtube Upload Processing and Queueing Theory"}]},{"@type":"WebSite","@id":"https:\/\/staging.opexlearning.com\/resources\/#website","url":"https:\/\/staging.opexlearning.com\/resources\/","name":"OpEx Learning","description":"Lean Six Sigma. Simplified.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/staging.opexlearning.com\/resources\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/staging.opexlearning.com\/resources\/#\/schema\/person\/9335b5223b67189b35bda7d6be11c3fd","name":"Uday Kawar"}]}},"_links":{"self":[{"href":"https:\/\/staging.opexlearning.com\/resources\/wp-json\/wp\/v2\/posts\/349"}],"collection":[{"href":"https:\/\/staging.opexlearning.com\/resources\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/staging.opexlearning.com\/resources\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/staging.opexlearning.com\/resources\/wp-json\/wp\/v2\/users\/12327"}],"replies":[{"embeddable":true,"href":"https:\/\/staging.opexlearning.com\/resources\/wp-json\/wp\/v2\/comments?post=349"}],"version-history":[{"count":0,"href":"https:\/\/staging.opexlearning.com\/resources\/wp-json\/wp\/v2\/posts\/349\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/staging.opexlearning.com\/resources\/wp-json\/wp\/v2\/media\/9278"}],"wp:attachment":[{"href":"https:\/\/staging.opexlearning.com\/resources\/wp-json\/wp\/v2\/media?parent=349"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/staging.opexlearning.com\/resources\/wp-json\/wp\/v2\/categories?post=349"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/staging.opexlearning.com\/resources\/wp-json\/wp\/v2\/tags?post=349"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}