Writable AI is the best AI writing assistant because it helps you improve your writing skills by providing feedback on your style, grammar, and spelling. It also makes suggestions on how to improve your writing. You can for sue use it for your academic papers, work emails, and even for your personal writing projects. If you want to increase your SEO ranking, Writable AI can also help you by optimizing your content for search engines.
Writable AI
“In software, the difference between the average software developer and the best is 50:1; Maybe even 100:1.” Steve Jobs
.
The Best v.s The Average
If you’re a software engineer, you would probably agree with what Steve Jobs said. The gap between the best and the average software engineers is incredibly large, much larger than most industries’.
However, there are still some industries that have similar talent distributions as software engineering does. The most familiar example is professional sports where the top 1% have much better performance than the average. And this talent gap is also reflected in atheletes’ salaries, which makes total sense. For example, one of the top MLB pitchers, Stephen Strasburg, received $38M in 2019, while the average salary in MLB was $4.3M.
But if salaries are a reflection of athletes’ contributions, why isn’t the distribution of software developers’ salaries look like athletes’?
.
Why Is It So Hard To Evaluate A Software Engineer?
In my opinion, the reason is that companies are not able to evaluate software engineers like how sports clubs evaluate their players. As most professional sports have very complete and objective statistics of players, managers can evaluate players quite accurately. Thus, clubs pay much higher for those who are excellent because everyone knows they’re good.
If one club doesn’t want to pay that much, other clubs will want to. Although there could be some differences between a player’s value and price sometimes, the gap between them will never be too big. This is just how the market works when information is very transparent.
On the other hand, the software development market is like a totally different world. You can’t really know if someone is a good software engineer unless you ever work with them closely for an extended period of time.
If you’re a programmer, you probably ever have a teammate that can produce things 10X, 50X compared the average productivity. And now think of if you’re boss’s boss will know he/she is a 10X engineer? Will your CTO and CEO know they are such a brilliant engineer and be willing to pay them 10X more than average to retain them?
Most likely no.
In reality, those 10X super engineers get a 10% pay raise and that’s it. The root cause is people don’t know how good they are except there direct coworkers.
.
Can We Use Programming Interview Results As A Proxy?
In fact, you can see how hard it is to evaluate a software engineer from the process of big tech companies. The most common technical interview process for programmers is to ask them to solve very challenging programming questions.
A lot of those questions are extremely hard to solve if an interviewee has never seen them before. That’s why many people choose to practice (or a bit of memorizing) a set of likely interview questions very intensively (also known as leetcoding).
Also, the ability to solve a very hard algorithmic problem in a short time does not highly correlate to one’s performance at work. What a software engineer is actually doing at work is just so different than solving those very-well defined programming problems. It’s not a desirable way software engineer’s capability.
Many problems one needs to be solved at work are very ambiguous. Knowing how to break down an ambiguous problem into a few well-defined programming subproblems and how to frame a business problem as a well-study research problem is far more crucial for being an effective engineer.
Even when many companies have already realized this recruiting process issue, they still prefer it. Why? It’s simply because there are no better ways to evaluate a software developer’s ability. Maybe this conventional interview process can only measure 20% of a software engineer’s work-related ability, it’s better than nothing. So you know how hard it is.
.
Difficult Evaluation Makes Salaries Distribution More Evenly
The reason I bring up the interview example above is to elaborate that it’s almost impossible to evaluate a software engineer in a scalable way. Now, I want to do a thought experiment to further elaborate it.
Let’s assume it’s as easy to measure a programmer’s ability as a professional athlete. If that assumption is true, your engineer coworkers should be fine with some people get $2M a year and some get $100K a year. Even if they all know each other’s salaries and how extreme the distribution is, they should still feel fair. Because they know those who get paid 20X are just 20X better than others. They deserve it.
But imagine if a company really makes its salary distribution like that, people will not feel unfair or even angry? I think they definitely will except those top performers’ close coworkers (those who work with them every day). That’s the point I want to make. It’s just too hard to evaluate it.
.
Conclusion
Because it’s virtually impossible to measure software engineers at scale and objectively. Companies prefer a safer way to distribute software engineers’ rewards, which might not reflect their productivity accordingly.
Explainability v.s Deep Learning
Explainability is like a ghost that has been hunting deep learning researchers for many years. You can hear many people ask AI researchers to provide more explainability for their deep learning models from time to time. However, we, human beings, seem to forget that we often also don’t know how to explain our behaviors, especially for system-1 tasks.
Maybe let me explain system-1 tasks and system-2 tasks a bit. This concept was first introduced in the book “Thinking, Fast and Slow”. System-1 tasks are done fast and unconsciously such as recognizing if it’s a dog or a cat, understanding a simple sentence, and driving a car on an empty road.
On the contrary, system-2 tasks are done slowly and consciously, for example, digging into your memory to recognize a sound, solving 17 × 24, and giving someone your phone number. So, when your brain is performing a task, like identifying if an object is a dog, it takes many raw inputs (pixies), feeds those into many layers of billions of neurons, and then it draws a conclusion.
And if I ask you why you think it is a dog instead of wolf, you will probably need to spend quite a long time to explain why you think so. You might come up with some reasons (e.g. its mouse is sharper and so on), but you probably didn’t make your decision because of any of those.
And this kind of task (system 1) is what exactly deep learning models are good at for now. In case you are curious, deep learning models for object classification usually have a very similar mechanism to how human brains do this. There are also different layers of neurons, and the deeper the layer is, the more abstract concepts it can understand, and it makes a decision in the very last layer.
As we hardly can even explain ourselves, why are we always asking machines to explain themselves? I actually spent some time thinking about this question. I believe there are two different reasons for this.
1. Lack of Dark Knowledge
Even deep learning models can outperform humans in many different tasks, but they sometimes make very unbelievable mistakes that humans will never make — for example, classifying a dog as an airplane. If a human fails to classify a dog as a dog, they probably will guess it is a cat or a wolf, which is not too far from a dog in terms of common sense. We call this kind of common sense “dark knowledge”.
And whenever people see such mistakes made by a model, they just think it is crazy, it’s a black box, and want to ask for an explanation. The major problem for this is not about explainability, and it is how we can let machines learn such dark knowledge and how we can represent it in a machine learnable way.
2. Lack of Consistency
Machine learning models usually can make very different predictions for certain cases when they were trained on the same data. To be concrete, 10 different models might have very similar accuracies for the test data, but they might be good at and bad at very different kinds of case. This means they see things very differently. Some cases may be very difficult for one model but extremely easy for another. (Maybe two models will have a high agreement if they use similar architectures. I have not done much research on this yet.)
They are not very consistent, which is quite different from humans. The images which are hard for humans often have very similar attributes. That’s why we think humans are much more interpretable than deep learning models. In my opinion, the main problem for this is not lack of explainability, but we just function differently from models.
Solutions?
We can actually sum up two reasons above to one concept, which is we want models to be like us. I do believe this is something related to our evolution. Humans prefer humans than any other species, and having a machine that acts like humans makes us feel safe. But the problem is if machines can already outperform us, why should we force than to be like us?
There are typically two different angles for this among AI researchers. One is we should not force any human biases to models, and they should just learn every thing from data or rewards by themselves. The best example for this is AlphaZero, which learns everything from playing with itself, and eventually became the strongest Go and Chess player of all time. It even came up with some moves that humans had ever thought of, and humans ended up learning a lot from it.
Another side of the argument is that we should introduce human knowledge to machines. I do think it makes sense (somehow), but the problem with this approach is that we have not yet found good ways to represent human knowledge and implement it. What most people currently do is using symbolic AI approaches (using many handcrafted rules to guide machines). This makes models less flexible because you will need experts to handcraft rules for every different problem, and humans can also introduce wrong assumptions to machines.
(A discussion between Yann LeCun and Christopher Manning might help you understand those two views more.)
Encoding human knowledge into models in a more machine learning way is what AI researchers should overcome in the next decade, and it might be the key to progress our models from performing system-1 tasks to system-2 ones. And this line of research has not been valued in academia that much yet. If you are interested in this topic, you probably will not want to miss an epic talk given by Yoshua Bengio at NeurIPS 2019.
I write articles about deep learning, AI, and natural language understanding so follow me if you are interested.