Part of feedburner.com
Possibly pithy insights into computer performance analysis and capacity planning based on the Guerrilla <a href="http://www.perfdynamics.com/books.html">series of books</a> and <a href="http://www.perfdynamics.com/Classes/schedule.html">training classes</a> provided by <a href="http://www.perfdynamics.com/">Performance Dynamics Company</a>.
Amazingly, performance analysis (done right) is often not technology-dependent, per se. The right abstraction can remain invariant into perpetuity. For example, my 2025 analysis of the GPT Efficient Computer Frontier (ref. 8 in paper 1) is based partly on my 1989 analysis of virtual memory thrashing in paper 53. Same paradigm; vastly different context.
The connection between the two papers is by no means obvious but, quite striking (not to mention satisfying) when you realize it.
This is my general solution, computed in Mathematica: a functional programming language. Scroll through the PDF for details.
This topic goes back to my M.Sc. thesis, when my supervisor, Prof. Christie Eliezer (2nd from left in the photo), who was one of Dirac's very few research students, told me about the Dirac-Sommerfeld paradox. He thought it might be pertinent to my thesis.
Being a physics "genius", I didn't pursue it because I assumed it had been resolved by more modern developments like, Quantum Field Theory — a subject that neither Eliezer nor I knew much about. I was wrong. Very wrong!
While reading it I had some thoughts that turned out to be too long for tweeting a response. So, here I am revisiting my traditional blog, mostly for the formatted space it provides, but also to remind myself that it still lives!
The Article The article contains a massive amount of very admirable work (~1 year of effort, including benchmark code development) that is exceptionally well written, considering all the details that are covered. I didn't come across any typos or misspellings, either. There are actually TWO articles in one: an extensive summary analysis (not an Executive Summary) and an equally long Appendix containing the detailed measurements obtained from each disk model that was benchmarked.
Quite apart from my subsequent remarks below, the inferences about internal disk structure drawn from the microbenched timing-data are quite remarkable. And the beautiful plots generated from those data (e.g., mapping thousands of sector defects) are something to behold. Understandably, HDD manufacturers are not keen to include such plots in their marketing collateral.
The author states the following as his prime motivations for developing the extended microbenchmark algorithms.
"I had initially set out to detect the number of platters in a disk without opening up a disk, but in modern disks this simple-sounding task requires measuring several other properties before inferring the count of recording surfaces."Although a truly laudible effort, that is largely driven by scientific "curiosity", I can't help but wonder if the author is straining at a gnat."Many of the prior works had practical applications in mind (as opposed to pure curiosity), and were willing to trade measurement speed for a small loss in accuracy. Although [my microbenchmark] algorithms are not fundamentally different, prior work likely avoided exploring these because these measurements can take many hours and there is little practical benefit for applications such as performance modelling or improving data layouts in filesystems."
With a bit of perseverance, I could find several corresponding drive specs online. And, although manufactuers never lie (smirk) the search terms can vary widely.
Platters = disks = discs... Surfaces = heads = actuators...Applying the author's terminology:
Example Geometries The following disk models were compared against their respective manufacturer specs that I was able to find online.
The author infers the same number of WD S25 used surfaces (3) based on his benching data which, in itself, is quite remarkable. On the other hand, this same information is generally included in the disk manufacturer specs (or similar docs). Worst case, you could actually contact the manufacturer and they would very likely tell you; especially if they thought they could sell you some disks.
Bitter Pill However, even when you determine all these interesting extended disk metrics (measured, calculated or specified), you are no better off when it comes to using that information for better storage performance — as the author admits (see quote above) — or predicting future HDD internal geometries. Just like with modern multicore microprocessors,
Computer manufacturers of all stripes are in business to make a profit and to avoid being eaten by the competition. They will do whatever it takes (good, bad or ugly), whenever it's deemed necessary, to maintain or improve their market position. An unfortunate casuality of this ongoing commercial warfare is that any painstakingly acquired scientific analysis can become an historical footnote overnight.
Performance Modeling On the topic of performance modeling of disks (to which the author also alludes), his article is a good reminder of the mind-boggling complexity of data layout in modern disks. And that's essentially just the static picture. Along with that goes the complex dynamics of local request-chain caching and optimization.
In that vein, it's a fool's errand to try and model individual modern disks. In principle, you would need to construct a simulation model to handle all the potential transient behaviors. But that, in turn, requires deciphering the embedded controller algorithms, which are not only highly complex but proprietary. You'll never figure them out and no manufacturer will ever reveal them. So, a performance simulation is actually a non-starter.
On a more positive note, there is a surprisingly counterintuitive technique for constructing accurate performance models of collective disks, like SANs, using the PDQ (Pretty Damn Quick) performance analyzer. That's something I demonstrate in my Guerrilla Capacity and Performance (GCAP) classes.
Related Posts
All modern computer systems, no matter how complex, can be thought of as a directed graph of individual buffers that hold requests until to be serviced at a shared computational resource, e.g., a CPU or disk. Since a buffer is just a queue, any computer infrastructure, from your laptop up to Facebook.com, can be represented as a directed graph of queues.
The directed arcs or arrows in such a graph correspond to workflows between different queues. In the parlance of queueing theory, a directed graph of queues is called a queueing network model. PDQ is a tool for predicting performance metrics such as, waiting time, throughput, optimal user-load.
Two major benefits of using PDQ are:
New Featues
Maintenance Changes The migration of Python from 2 to 3 has introduced maintenance complications for PDQ. Python 3 may eventually be accommodated in future PDQ releases. Perl maintenance has ended with PDQ release 6.2, which to remain compatible with the Perl::PDQ book (2011).
As shown in the above diagram, any modern computer system can be thought of as a directed graph of individual buffers where requests wait to be serviced at some kind of shared computational resource, e.g., a CPU or disk. Since a buffer is just a queue, any computer infrastructure, from a laptop to Facebook, can be represented as a directed graph of queues. The directed arcs or arrows correspond to workflows between the different queues. In the parlance of queueing theory, a directed graph of queues is called a queueing network model. PDQ is a tool for calculating the performance metrics, e.g., waiting time, throughput, optimal load, of such network models.
Some example PQD applications include models of:
Converting between various time zones, including UTC time, is both a pain and error-prone. A better solution is to use absolute time. Thankfully, Unix provides such a time: the so-called Epoch time, which is the integer count of the number of seconds since January 1, 1970.
Timestamps are both very important and often overlooked: the moreso in the context of performance analysis, where time is the zeroth metric. In fact, my preferred title would have been, Death to Time Zones but that choice would probably have made it harder to find the main point here, later on, viz., how to use the Unix epoch time.
Although there are web pages that facilitate such time conversions, there are also shell commands that are often more convenient to use, but not so well known. With that in mind, here are some examples (for future reference) of how to convert between human-time and Unix time. Examples The following are the bash shell commands for both MacOS and Linux.
MacOS and BSD Optionally see the human-readable date-time:
[njg]~% date
Tue Feb 11 10:04:32 PST 2020
[njg]~% date +%s
1581444272
[njg]~% date -r 1581444272
Tue Feb 11 10:04:32 PST 2020
Linux Optionally see the human-readable date-time:
[njg]~% date
Tue Feb 11 10:04:32 PST 2020
[njg]~% date +%s
1581444272
[njg]~% date -d @1581444272
Tue Feb 11 18:04:32 UTC 2020
In the meantime, over the past 20 years, the nature of the book itself has become progressively more digital. The ability to render the book-block on digital devices as an "e-book" has made printed hardcopies optional. Although purely digital e-books make reading more ubiquitous, e-book file formats and display quality varies across devices, i.e., laptops, phones tablets, and various e-readers. Any reduction in display quality is offset by virtue of e-books being able to include user interaction and even animation: features entirely beyond the printed book. In that sense, there really has been a revolution in the publishing industry: books are no longer about books, they're now about media.
Recently, I became aware of an undergraduate calculus textbook that makes powerful use of animation and audio. The author is a academic mathematician and he published it on YouTube! Is that a book or a movie? Somehow, it's a hybrid of both and, indeed, I wish I'd been able to learn from technical "books" like that. Good visuals and animations can make difficult technical concepts much easier to comprehend. Progressive as all that is, when it comes to technical e-books, I'm not aware of any single authoring tool that can match the quality of LaTeX, let alone incorporate user-interaction and animation. If such a thing did exist, I would be all over it. And Markdown doesn't cut it. But, digital authoring tools are continually evolving.
Matt Fleming (@fleming_matt on Twitter), the author of How to Build a Performance Testing Stack From Scratch, opted to use a static e-book format—not because it produces the most readable result, but because it is the best way to reach a wider audience at lower cost than a more expensive print publisher. The e-publisher in this case is (the relatively unknown) Ministry of Testing Ltd in Brighton, UK and is available on Amazon for the Kindle reader. I was also able to read it using iBooks on Mac OS X.
The range of topics covered is very extensive. I've included the Table of Contents here because it is not viewable on Amazon:
Part 1The overall e-book presentation of "Performance Testing Stack" seems underdeveloped. Most topics could have been greatly expanded. But, as Matt informed me, this is probably due to the content amounting to a concatenation of previously written blog posts. As I said earlier, there are no shortcuts to writing well. The paucity of detail, however, is offset by the shear enthusiasm the Matt brings to his writing: an aspect that deserves separate acknowledgement because performance testing is a very complex subject which can otherwise appear dry and mind-boggling to the uninitiated. And it's the uninitiated that Matt wants to reach. He has written this book in order to encourage the uninitiated reader to seriously consider entering the field.Part 2
- Step 1: Identify Stakeholders
- Step 2: Identify What to Measure
- Step 3: Test Design
- Step 4: Measuring Test Success and Failure
- Step 5: Sharing Results
Part 3 The Benchmark Hierarchy Picking Tests Validating Tests Part 4
- Understanding Statistics
- Latency
- Throughput
- Statistical Significance
Part 5
- Use a Performance Framework
- Ensure the Test Duration is Consistent
- Order Tests by Duration
- Keep Reproduction Cases Small
- Setup The Environment Before Each Test
- Make Updating Tests Easy
- Errors Should Be Fatal
- Format
- Use Individual Sample Data
- Detecting Results in the Noise
- Outliers
- Result Precision
- If All Else Fails Use Test Duration
- Delivering Results
Some points that could have been developed further, include:
Conversely, Matt's enthusiasm may have gone a bit overboard in his choice of title. The book promises:
This book will walk you through designing and building a performance testing stack from scratch; step by step from planning and running performance tests through to understanding and analysing the results. If you’re new to performance testing or looking to expand your understanding of this topic then this book is for you!Unfortunately, this book doesn't provide enough details to actually build a test stack—which would've been very cool. Rather, it presents a comprehensive overview of all the major concepts that one needs to absorb in order to develop a running performance testing stack. But even with the more limited scope, this book is still important because, off hand, I don't know of any other source where one can be introduced to performance testing without drowning in a sea of terminology, procedures and architectures.
Ultimately, this e-book is a great starting point for newbies, as well was being a good reminder for seasoned testers about what should be done in good performance tests.
DSConf'19 - Distributed Systems Conference (scroll down)
Pune, India
11am IST
February 16
I'm very much looking forward to this event and I thank @ShrivedAgashe for the invitation.
The following highlights indicate the kind of thing you'll learn. Most especially, how to make better use of all that monitoring and load-testing data you keep collecting.
See what Guerrilla grads are saying about these classes. And how many instructors do you know that are available for you from 9am to 9pm (or later) each day of your class?
Who should attend?
As usual, Sheraton Four Points has bedrooms available at the Performance Dynamics discounted rate. The room-booking link is on the registration page.
Tell a colleague and see you in September!
The details concerning how you can do this kind of cost-benefit analysis for your cloud applications will be discussed in the upcoming GCAP class and the PDQW workshop. Check the corresponding class registration pages for dates and pricing.
Update of Oct 2018: Wow! MathJax performance is back. Clearly, whinging is the most powerful performance optimizer. :)
The 2-parameter USL model The original USL model, presented in my GCAP book and updated in the blog post How to Quantify Scalability, is defined in terms of fitting two parameters $\alpha$ (contention) and $\beta$ (coherency). \begin{equation} X(N) = \frac{N \, X(1)}{1 + \alpha \, (N - 1) + \beta \, N (N - 1)} \label{eqn: usl2} \end{equation}
Fitting this nonlinear USL equational model to data requires several steps:
The 3-parameter USL model Going back to equation (\ref{eqn: usl2}), let's just consider the simplest case where scaling is linear-rising, as would be the case for ideal parallelism. In the linear region, where $\alpha = \beta = 0$, equation (\ref{eqn: usl2}) simplifies to \begin{equation} X(N) = N \, X(1) \label{eqn: usl1} \end{equation}
In other words, the overall throughput $X(N)$ increases in simple proportion to $N$. The "single-user" throughput, $X(1)$, doesn't change and therefore acts like a constant of proportionality.
But what happens when we don't know the value of $X(1)$? That means the $X(1)$ factor in equations (\ref{eqn: usl2}) and (\ref{eqn: usl1}) is undefined. We might denote this situation by writing
\begin{equation} X(N) = N \, ? \label{eqn: uslx} \end{equation}
Of course, that makes no sense, mathematically speaking. As already mentioned, the conventional way out of this situation is to estimate the value of $X(1)$ using mathematical interpolation. But here's the epiphany.
Rather than using the more complicated interpolation procedure, we can simply appeal to statistical regression! Yes, that's right, we treat the USL equation as a conventional multivariate statistical model. After all, we're already using nonlinear statistical regression to determine the $\alpha$ and $\beta$ parameters. More importantly, since statistics is not math, we can replace equation ($\ref{eqn: uslx}$) with a statement about correlation, rather than strict equality. In statistical models, that's accomplished by introducing another parameter (I'll call it $\gamma$, since that's the third letter of the Greek alphabet) to replace the question mark in equation ($\ref{eqn: uslx}$), namely
\begin{equation} X(N) = N \, \gamma \label{eqn: uslg} \end{equation}
The new parameter $\gamma$ is just a constant of proportionality that represents the slope of the line associated with ideal parallel scaling. See the plots below.
And here's a little piece of magic. If we choose $N = 1$ in equation ($\ref{eqn: uslg}$), it becomes $X(1) = \gamma$. So, when the $\gamma$ parameter is determined by statistical regression, it also tells us the estimated value of $X(1)$, whether it was measured or missing. In other words, we don't need to do any explicit interpolation because the nonlinear regression procedure does it automatically by fitting the third parameter.
Equation (\ref{eqn: usl2}) is now replaced by a 3-parameter version of the USL model: \begin{equation} X(N) = \frac{N \, \gamma}{1 + \alpha \, (N - 1) + \beta N \, (N - 1)} \label{eqn: usl3} \end{equation}
Unlike the 2-parameter USL, equation (\ref{eqn: usl3}) can be fitted directly to your throughput measurements without the need to do any data normalization or interpolation. The following examples show the results of fitting the 3-parameter USL model.
Load-test data These are load-test data and the "single-user" throughput was measured as $X(1) = 955.16$ per unit time. The 3-parameter USL fit is summarized in the following plot.
The fitted value of $\gamma = 995.65$, which is the estimated value of $X(1)$. It can also be regarded as the slope of the linear-rising throughput indicated by the sloping red line on the left of the plot.
Production data These data are from a continuously running production system and thus, no $X(1)$ was ever produced.
The fitted value of $\gamma = 3.22$ is also equivalent to the estimated value of $X(1)$. Similarly, it can be regarded as the slope of the linear-rising throughput on the left of the plot. Interestingly, in these data, $\alpha = 0$, while $beta$ is non-zero. That suggests there is no significant contention in the workload but there is some data exchange coherency at play.
One word of caution. Fitting the 3-parameter USL can be more sensitive to the actual data, especially with a large number of production data scatter points. I'll go into all this, and more, in the upcoming Guerrilla training classes.
Here's a mnemonic tabulation based on dishes and bowls:
Hopefully this makes amends for the more complicated explanation I wrote for CMG back in 2009 entitled: "Mind Your Knees and Queues: Responding to Hyperbole with Hyperbolæ", which I'm pretty sure almost nobody understood.
Exposing the Cost of Performance
Hidden in the Cloud
Neil Gunther
Performance Dynamics, Castro Valley, California
Mohit Chawla
Independent Systems Engineer, Hamburg, Germany10am Pacific Time on June 19, 2018
Whilst offering lift-and-shift migration and versatile elastic capacity, the cloud also reintroduces an old mainframe concept—chargeback—which rejuvenates the need for performance analysis and capacity planning. Combining production JMX data with an appropriate performance model, we show how to assess fee-based EC2 configurations for a mobile-user application running on a Linux-hosted Tomcat cluster. The performance model also facilitates ongoing cost-benefit analysis of various EC2 Auto Scaling policies.
The strength of the model turns out to be its explanatory power, rather than prediction, per se. However, with the correct explanation of the performance problem in hand (which also proved that all other guesses were wrong), this model correctly predicted a 300% reduction in application response time for essentially no cost. Modeling doesn't get much better than this.
Footnotes
Under pressure to consolidate resources, usually driven by management and especially regarding processor capacity, there is often an urge to "use up" any idle processor cycles. Idle processor capacity tends to be viewed like it's whitespace on a written page—just begging to be filled up.
The logical equivalent of filling up the "whitespace" is absorbing idle processor capacity by migrating applications that are currently running on other servers and turning those excess servers off or using them for something else.
The blindspot, however, is that idle processor capacity is not like whitespace and the rush to absorb it is likely to have unintended consequences. The reason is that performance metrics are neither isolated nor independent of one another. Most metrics are:
For example, application response time depends nonlinearly on processor utilization. In the above case, it may be have been forgotten that the processor utilization must be kept low in order for the application to meet its response time SLA. A lot of idle processor cycles can appear to be unused processor capacity only because there is no obvious warning sign that low CPU is a necessary condition for correct application performance.
Even on a printed page, whitespace is not usually an invitation to start scribbling on it. Similarly, a notification equivalent to "This page intentionally left blank" would be a useful reminder in the context of potential application migration.
Of course, any page that says "This page left blank" isn't blank, but that's a topic for a different discussion. :)
Who should attend?
Some course highlights:
Register online. Early-bird discounts run through the end of August.
As usual, Sheraton Four Points has bedrooms available at the Performance Dynamics discounted rate. Link is on the registration page.
Also see what graduates are saying about these classes.
Tell a colleague and see you in September!
This year is the centenary of A. K. Erlang's paper [1] on the determination of waiting times in an M/D/m queue with $m$ telephone lines.* Today, M/M/m queues are used to model such systems as, call centers [3], multicore computers [4,5] and the Internet [6,7]. Unfortunately, those who should be using M/M/m models often do not have sufficient background in applied probability theory. Our remedy defines a morphing approximation† to the exact M/M/m queue [3] that is accurate to within 10% for typical applications‡. The morphing formula for the residence-time, $R(m,\rho)$, is both simpler and more intuitive than the exact solution involving the Erlang-C function. We have also developed an animation of this morphing process. An outstanding challenge, however, has been to elucidate the nature of the corrections that transform the approximate morphing solutions into the exact Erlang solutions. In this presentation, we show:
- The morphing solutions correspond to the $m$-roots of unity in the complex $z$-plane.
- The exact solutions can be expressed as a rational function, $R(m,z)$.
- The poles of $R(m,z)$ lie inside the unit disk, $|z| < 1$, and converge around the Szego; curve [8] as $m$ is increased.
- The correction factor for the morphing model is defined by the deflated polynomial belonging to $R(m,z)$.
- The pattern of poles in the $z$-plane provides a convenient visualization of how the morphing solutions differ from the exact solutions.
* Originally, Erlang assumed the call holding time, or mean service time $S$, was deterministic with unit period, $S=1$ [1,2]. The generalization to exponentially distributed service periods came later. Ironically, the exponential case is easier to solve than the apparently simpler deterministic case. That's why the M/D/1 queue is never the first example discussed in queueing theory textbooks.
† The derivation of the morphing model is presented in Section 2.6.6 of the 2005 edition of [4], although the word "morphing" is not used there. The reason is, I didn't know how to produce the exact result from it, and emphasizing it would likely have drawn unwarranted attention from Springer-Verlag editors.
By the time I was writing the 2011 edition of [4], I was certain the approximate formula did reflect the morphing concept in its own right, even though I still didn't know how to connect it to the exact result. Hence, the verb "morphs" timidly makes its first and only appearance in the boxed text following equation 4.61.
‡ The relative error peaks at 10% for $m \sim 20$ and $\rho \sim 90\%$, then peaks again at 20% for $m \sim 2000$ and the servers running 99.9% busy. However, the rate of increase in peak error attenuates such that the maximum error is less than 25%, even as $m \rightarrow \infty$ and $\rho \rightarrow 100\%$. A plot of the corresponding curves gives a clearer picture. This behavior is not at all obvious. Prior to this result, it could have been that the relative error climbed to 100% with increasing $m$ and $\rho$.
References
"As GitHub approaches 700 employees, with more than $200M in ARR, accelerating growth, and more than 20 million registered users, I'm confident that this is the moment to find a new CEO to lead us into the next stage of growth. ....."The Original Analysis In 2013, a Redmonk blogger claimed that the growth of GitHub (GH) users follows a certain type of diffusion model called Bass diffusion. Here, growth refers to the number of unique user IDs as a function of time, not the number project repositories, which can have a high degree of multiplicity.
In a response, I tweeted a plot that suggested GH growth might be following a power law, aka scale free growth. The tell-tale sign is the asymptotic linearity of the growth data on double-log axes, which the original blog post did not discuss. The periods on the x-axis correspond to years, with the first period representing calendar year 2008 and the fifth period being the year 2012.
Scale free networks can arise from preferential attachment to super-nodes that have a higher vertex degree and therefore more connections to other nodes, i.e., a kind of rich-get-richer effect. Similarly for GH growth viewed as a particular kind of social network. The interaction between software developers using GH can be thought of as involving super-nodes that correspond to influential users attracting prospective GH users to open a new account and contribute to their project.
On this basis, I predicted GH would reach 4 million users during October 2013 and 5 million users during April 2014 (yellow points in the Linear axes plot below). In fact, GH reached those values slightly earlier than predicted by the power law model, and slightly later than the dates predicted by the diffusion model (modulo unreported errors in the data).
Since 2013, new data has been reported so, I extended my previous analysis. Details of the respective models are contained in the R script at the end of this post. In the Linear axes plot below, both the diffusion model and power model essentially form an envelope around the newer data: diffusive on the upper side (red curve) and power law on the lower side (blue curve). In thise sense, it could be argued that the jury is still out on which model offers the more reliable predictions.
However, there is an aspect of the diffusion model that was overlooked in 2013. It predicts that GH growth will eventually plateau at 20 million users in 2020 (the 12th period, not shown) because it is a type of logistic function that has a characteristic sigmoidal or 'S' shape. The beginnings of this leveling off (the top of the 'S') is apparent in the 10th period (i.e., 2017). By contrast, the power law model predicts that GH will reach 23.65 million users by the end of 2017 (yellow point). Whereas the two curves envelope the more recent data in periods 6–9, they start to diverge significantly in the 10th period.
"GitHub is not the only player in the market. Other companies like GitLab are doing a good job but GitHub has a huge head start and the advantage of the network effect around public repositories. Although GitHub’s network effect is weaker compared to the likes of Facebook/Twitter or Lyft/Uber, they are the default choice right now." —GitHub is Doing Much Better Than Bloomberg ThinksAlthough there will inevitably be an equilibrium bound on the number of active GH users, it seems unlikely to be as small as 20 million, given the combination of GH's first-mover advantage and its current popularity. Presumably the private investors in GH also hope it will be a large number. This year will tell.
# Data source ... https://classic.scraperwiki.com/scrapers/github_users_each_year/
#LINEAR axes plot
plot(df.gh3$index, df.gh3$users, xlab="Period (years)",
ylab="Users (million)", col="gray",
ylim=c(0, 3e7), xaxt="n", yaxt="n")
axis(side=1, tck=1, at=c(0, seq(12,120,12)), labels=0:10,
col.ticks="lightgray", lty="dotted")
axis(side=2, tck=1, at=c(0, 10e6, 20e6, 30e6), labels=c(0,10,20,30),
col.ticks="lightgray", lty="dotted")
# Simple exp model
curve(coef(gh.exp)[2] * exp(coef(gh.exp)[1] * (x/13)),
from=1, to=108, add=TRUE, col="red2", lty="dot dash")
# Super-exp model
curve(49100 * (x/13) * exp(0.54 * (x/13)),
from=1, to=120, add=TRUE, col="red", lty="dashed")
# Bass diffusion model
curve(21e6 * ( 1 - exp(-(0.003 + 0.83) * (x/13)) ) / ( 1 + (0.83 / 0.003) * exp(-(0.003 + 0.83) * (x/13)) ),
from=1, to=120, add=TRUE, col="red")
# Power law model
curve(10^coef(gh.fit)[2] * (x/13)^coef(gh.fit)[1], from=1, to=120, add=TRUE,
col="blue")
title(main="Linear axes: GitHub Growth 2008-2017")
legend("topleft",
legend=c("Original data", "New data", "Predictions", "Exponentital", "Super exp", "Bass diffusion", "Scale free"),
lty=c(NA,NA,NA,4,2,1,1), pch=c(1,19,21,NA,NA,NA,NA),
col=c("gray", "black", "yellow", "red", "red", "red", "blue"),
pt.bg = c(NA,NA,"yellow",NA,NA,NA,NA),
cex=0.75, inset=0.05)
Presenter: James Brady (co-author: Neil Gunther)
Session Number: 436
Subject Area: APM
Session Date: Wed, November 9, 2016
Session Time: 1:00 PM - 2:00 PM
Session Room: PortofinoB
The motivation for this work harks back to a Guerrilla forum in 2014 that essentially centered on the same topic as the title of our paper. It was clear from that discussion that commenters were talking at cross purposes because of misunderstandings on many levels. I had already written a much earlier blog post on the key queue-theoretic concept, viz., holding the $N/Z$ ratio constant as the load $N$ is increased, but I was incapable of describing how that concept should be implemented in a real load-testing environment.
On the other hand, I knew that Jim Brady had presented a similar implementation in his 2012 CMG paper, based on a statistical analysis of the load-generation traffic. There were a few details that I couldn't quite reconcile in Jim's paper but, at the CMG 2015 conference in San Antonia, I suggested that we should combine our separate approaches and aim at a definitive work on the subject. After nine months gestation (ugh!), this 30-page paper is the result.
Although our paper doesn't contain any new invention, per se, the novelty lies in how we needed to bring together so many disparate and subtle concepts in precisely the correct way to reach a complete and consistent methodology. The complexity of this task was far greater than either of us had imagined at the outset. The hyperlinked Glossary should help with the terminology, but because there are so many interrelated parts, I've put together the following crib notes in an effort to help performance engineers get through it (since they're the ones that most stand to benefit). The key results of our paper are indicated by ◀
The right-hand side of each equation representing a version of Little's law is written vertically in the order $R, W, S$, which matches the expression $R=W+S$ for the mean residence time, viz., the sum of the mean waiting time ($W$) and the mean service time ($S$).
The letters on the left-hand side: $Q, L, U$ (reading vertically) respectively correspond to the queueing metrics: queue-length, waiting-line length, and utilization, which can be read as the word clue.
Incidentally, the middle formula is the version that appears in the title of John Little's original paper:
J. D. C. Little, ``A Proof for the Queuing Formula: L = λ W,''
Operations Research, 9 (3): 383—387 (1961)