Observational Hazard

Many have compared the advancements in LLMs for software development to the improvements in abstraction that came with better programming languages.

But there’s a key difference. While the end product of software development, a binary, has never drastically changed its form (though its distribution has radically changed), the intermediate product did with programming language transitions.

The intermediate product is the source code itself. The intermediate goal of a software development project is to produce robust maintainable source code. The end product is to produce a binary. New programming languages changed the intermediate product. When a team changed from using assembly, to C, to Java, it drastically changed its intermediate product. That came with new tools built around different language ecosystems and different programming paradigms and philosophies. Which in turn came with new ways of refactoring, thinking about software architecture, and working together.

LLMs don’t do that in the same way. The intermediate product of LLMs is still the Java or C or Rust or Python that came before them. English is not the intermediate product, as much as some may say it is. You don’t go prompt->binary. You still go prompt->source code->changes to source code from hand editing or further prompts->binary. It’s a distinction that matters.

Until LLMs are fully autonomous with virtually no human guidance or oversight, source code in existing languages will continue to be the intermediate product. And that means many of the ways that we work together will continue to be the same (how we architect source code, store and review it, collaborate on it, refactor it, etc.) in a way that it wasn’t with prior transitions. These processes are just supercharged and easier because the LLM is supporting us or doing much of the work for us.

As an aside, I think there may be an increased reason to use dynamic interpreted languages for the intermediate product. I think it will likely become mainstream in future LLM programming systems to make live changes to a running interpreted program based on prompts. The hit run refresh cycle will be reduced to zero. That will be vibe coding to the max. Perhaps this is already more mainstream now than I’m aware.

My fifth book, Computer Science from Scratch: Building Interpreters, Art, Emulators, and ML in Python was recently published by No Starch Press. It’s a great book for intermediate or advanced Python programmers who have some gaps in their CS knowledge and want to learn from a code-centric, project-focused approach. It includes seven fun, educational projects like building a BASIC interpreter and an NES emulator. You can learn more about the book itself on computersciencefromscratch.com. In this post I’m going to tell you about the development process. Everything from selecting the topics, to writing the manuscript, finding a publisher, and getting it through to the finish line is covered here.

My most successful prior book was Classic Computer Science Problems in Python which was published by Manning in 2019. That book sold tens of thousands of copies around the world and was translated into eight different languages. I think it hit a gap in the market. There were many great introductory books in Python. There were not many great intermediate books in Python. It was designed for people who already knew Python and wanted to learn some ideas from the world of computer science. It covered some data structures and algorithms topics like graph algorithms and several topics from the world of artificial intelligence like adversarial search and neural networks. All of the topics came with fully working code built from scratch in Python with no external libraries.

After Classic Computer Science Problems in Java was published (my 4th book), I thought, “Do I have anything new to say?” I realized I did. I had accumulated interesting personal projects and projects that I used while teaching CS in a college setting that would make a nice compilation of introductory material to other realms in CS. They were less about classic algorithms than the Classic Computer Science Problems series and more about the layers of the software stack. I also thought I should build on my success with a previous intermediate Python book. So, what was originally titled Fun Computer Science Projects in Python, was born. It would be for the same general audience as Classic Computer Science Problems in Python but would have no direct content overlap. It would feature larger, more fun, projects. Eventually, before release, the publisher, No Starch Press, came up with a better title: Computer Science from Scratch. How does a programming language work? How does a computer interpret instructions? These are the kinds of questions I want to answer for my intermediate Python readers.

I started writing Computer Science from Scratch four-and-a-half years before it was published. Before I wrote any of the text, I started writing the code for several of the book’s projects. As I mentioned, I adapted projects from my college teaching career or personal projects for many of them. Most of the projects were not originally written in Python, so I found myself porting them and cleaning them up.

For example, I taught a class called Emerging Languages at Champlain College in which we used modern languages like Go, Swift, and Clojure as a lens to teach some programming language theory. One of the projects was building a Brainfuck interpreter in Clojure. Another was building a Tiny BASIC interpreter in Swift. I ported those projects to Python to create Part I: Interpreters.

One project in the book actually went the other direction. Chapter 3 is about building a program called Retro Dither that can take modern photographs, dither them so they look okay in black and white, and convert them to MacPaint format for display on a 1980s black & white Mac. Along the way you learn about dithering algorithms, file formats, and run-length encoding. I actually ended up porting that program from Python to Swift and releasing it on the Mac App Store. I wrote a blog post back in 2021 about that process as I was writing the book.

Chapter 4 is also connected to Swift. About a decade ago I became very enamored with Michael Fogleman’s Primitive project that turns photos into abstract vector art. I ended up writing an iOS app that does the same thing using a simpler algorithm of my own design. That program got ported to Python as Impressionist for Chapter 4 of the book. I think it’s amazing how cool some of the abstract art comes out. You’d think something sophisticated like a neural network must be behind it, but really it’s just some stochastic hill climbing.

Chapters 5 and 6 are projects from the world of emulation. Writing an emulator really helps you understand the hardware/software interface and how computers work at a lower level. You certainly learn something about instruction sets and computer architecture. I had been writing emulators in Swift, C, & C++ out of personal interest over the past decade. I ended up porting two of those projects, a CHIP–8 VM and an NES emulator, to Python for the book. Before porting the NES emulator to Python, I rewrote my C version with the simplest possible Picture Processing Unit (PPU) implementation since graphics were what really tripped me up when I was learning to write an NES emulator myself. It’s not super compatible, but its simplicity helps you understand what’s going on at a high level. I think the NES emulator chapter is the crown jewel of the book (it’s also the longest and hardest). To my knowledge, Computer Science from Scratch is the first book to cover how to write an NES emulator, although of course there are tutorials (of varying quality) online. By writing it in Python we make this popular hobbyist programming project accessible to a wider audience. However, it also means we don’t achieve full 60 FPS performance. We leave using Cython or something like it to achieve full performance as an exercise for the reader.

I will admit that I didn’t originally intend the last two chapters (7 & 8) on introductory machine learning using the KNN algorithm for the book. But as I was shopping the book around to publishers (I come up with all my book ideas myself and then shop them to publishers) it was suggested to me that any book introducing a reader today to the world of CS should have at least a little machine learning content. Actually even my book Classic Computer Science Problems in Swift from 2017 had two chapters in this vein (one on K-Means and one on neural networks). Luckily I had been teaching introductory artificial intelligence at the undergraduate level for the past few years and already had the perfect introductory ML project in mind. The code for naive KNN is basically trivial but it did save me some time to already know the project around it was pedagogically sound from having used it in the classroom. Like Impressionist from Chapter 4 it’s amazing that an algorithm as simple as KNN can get as great results as it does in these chapters, achieving 98% accuracy at classifying handwritten digits on a classic dataset.

I approached getting a deal for the book in what is generally an unusual way in the world of technical publishing. Instead of writing some of the book and getting a publisher on board before finishing it, I finished an entire first draft of the book and then shopped it around. Ultimately I had two offers. One from a very large, highly respected academic and trade publisher, and the other from No Starch Press. The very large publisher seemed as interested in me doing a video series about the book as in the book itself. No Starch Press, being much smaller and much more niche, was on the other end of the spectrum. They didn’t want video at all, just the book. Plus their royalty deal was better, I liked some of their prior books, and they were the best-selling publisher in the world of Python. So I went with them.

These shopping around processes can be intense. Some publishers take weeks or even months to get back to you. Unfortunately, I didn’t get to hear back from all of the publishers that were still considering the book and had to pull out of a couple of the processes to make a decision on time for the two that had already made offers. I also received a couple rejections. One niche technical publisher turned down the book because the first project, Brainfuck, has the F word in it. Hey I didn’t name the language… that’s just what the programming language is called! Another, very well known programming book publisher said they were interested but only if I would add LLM prompting content to each chapter. I had no interest in that as I thought it would get stale quickly.

I said I started working on the book four-and-a-half years before it was published. While that’s true, the whole book was actually written in several short bursts of activity for a couple months at a time. I’m a pretty fast writer. In fact my manuscript was done in July 2024 when I was shopping it around to publishers. It took over another year to get it out. That’s just what the publishing process with traditional publishers takes. A lot of that work was converting from markdown to Word templates, development editing, technical review, copyediting, making an index, and final layout. Tedious work that sometimes feels like busy work. Much of it done by the publisher with a ton of back-and-forth with the author. I will say the development editing on this book was particularly fruitful. My development editor, Nathan Heidelberger, really improved the book by taking the perspective of an intermediate Python programmer himself. Also thanks to technical reviewer Michael Kennedy of Talk Python fame who had some good technical catches.

Now that it’s done would I do anything differently? Well, I’m very satisfied with the result. I wrote every line of code and every word in the book by hand. I didn’t use any LLMs. In fact, widespread LLMs didn’t exist when I started the book. Of course it could have been written much faster if I started using one as the book progressed, but I think it wouldn’t have sounded as authentic if it had been. And I think authenticity in writing still comes through and connects with people. Another issue is that I considered self-publishing. I may still do that on a future book, but I ultimately decided against it for a few reasons: the legitimacy that comes from a traditional publisher vetting it (at least in my circles working in higher ed), the marketing power of a traditional publisher, and the style choices they would help with. Ultimately I think those style choices did make a difference: I like the title they came up with, the cover they came up with, and the stylistic changes the development editor suggested. But working with a traditional publisher also comes with a lot of bureaucracy, nitpicking, and giving up the majority of the profits from the book. You lose full control. So it’s a tradeoff for sure and not a clear cut decision even if you are fortunate enough to have an offer for a book deal.

If you read this far and you are an intermediate or advanced Python programmer please check out the book! It’s available now from Amazon, No Starch Press, and you can learn more about it on the website I created for it.

Update August 1, 2024

The original blog post below went a bit viral on Hacker News and garnered 60,000+ page views in just a couple days. Maybe that (or more likely other API partners complaining) had some impact at Yelp. They just sent the email that follows. I appreciate the apology and acknowledgement of the mistake. I also would like to clarify what I wrote in more words in the original post—Yelp is perfectly within their right to start charging for their API. They just did it in a very poor way providing us 4-days notice (1 business day really Friday->Monday).

Yelp's August 1st email (emphasis mine):

Earlier this month, we sent you an email about your Yelp Fusion API usage. That email gave developers until July 23rd to contact us if they want to continue using Yelp’s data for use in their app. We realize you might need more time and are extending your free access for an additional 90 days starting today. Your access should be available now.

We’re sorry for any inconvenience or frustration this abbreviated transition might have caused. Please respond to this email or contact us at api@yelp.com if you have any questions.

The Original Post

After 10 years, I have removed from sale my macOS app for figuring out where to eat, Restaurants. In this post I’ll explain what Restaurants was, how I developed it, and why it was removed from sale. It speaks to the challenges developers face when incorporating third-party APIs.

App Development Journey

The year was 2014 and Apple had recently released a beta of its new programming language, Swift. As an exercise for an early adopter of the language, I thought it would be fun to leverage my experience in a couple startups using the Yelp API to make a restaurant-search-specific Yelp client for the Mac.

I emailed Yelp, explained the app I intended to build, and got permission. The person in developer relations said since they didn’t have a native Mac client, they would approve my use of the Yelp API for Restaurants. In fact, without me specifically asking for it, they provided a 25,000 per day API call limit (a limit my app never even came close to reaching). It seemed they were in fact encouraging me to finish the app and release it.

This makes sense since Yelp limited severely how many results (20, later 50), reviews (3), and pictures (3) were available through their API. So, it’s not like my app could replicate the full experience of using the Yelp website or the Yelp iOS client. Instead, it served as a super fast, well-integrated with macOS, way to search for restaurants on your Mac with the option to go to the Yelp website to read more reviews and see more details. In other words, it was in many ways a traffic driver to the Yelp website.

I released the app and it sold moderately, but consistently. Over the decade it was available, I charged between $1.99 and $4.99 up-front for the app (a one-time purchase through the Mac App Store). Sometimes it was in the top 10 apps in the Lifestyle category on the Mac App Store. But it turns out you only need to sell a couple copies in a day to reach the top 10! The vast majority of sales were to US customers. I assume that’s either because the US is where Yelp is most popular or has to do with our restaurant culture, or the combination of the two.

I built the app using AppKit and some third-party Yelp libraries, including most recently CDYelpFusionKit. I was sure to integrate with as many macOS features as I could. For example, you could instantly call a restaurant on your phone using Continuity, add a Restaurant to your address book, or get directions in Apple Maps. You could share a Restaurant’s details with any of your social apps. And of course Restaurants could automatically detect your location for your search if you gave it permission. It was a small app, but it did the one thing it was meant to do, search for restaurants, well. I used it myself every time I thought about going out somewhere new to eat.

One of the most popular design decisions I made was to represent different categories of restaurants with different emojis. For example, traditional American restaurants would have the turkey emoji 🦃 next to their name. This allowed folks to quickly scan the search results for their cuisine of choice. You can see what this looked like in one of the screenshots on the Restaurants website. It was actually my wife’s idea. With her help, we figured out by hand what emojis to use for all of the many restaurant categories that Yelp provides.

Over the years as Yelp added more features to the Yelp API and rebranded it as Yelp Fusion, I added those features to Restaurants. In the most recent big update, version 3, I better-integrated Apple Maps for a more seamless view of your search results geographically.

Users loved the most recent version, Restaurants 3. Its reviews were all high. One reviewer wrote on the Mac App Store, “I’m new to the Mac world, but I just love this app. It’s beautifully designed, easy and flexible to use, and infiniitely helpful.”

The 4-Day Deadline

So, why did I remove Restaurants from sale? Well on July 19, 2024 I received the following email:

Hello API user,

We appreciate you signing up for and trialing the Yelp Fusion API.

Your API usage is higher than lots of other Yelp Fusion developers and we would like to learn more about how you’re integrating the Fusion API into your platform. Please share screenshots which represent how you implemented the Yelp Fusion API, display Yelp content, and clearly attribute Yelp to your users.

To continue using Yelp Fusion commercially, we need to complete a license agreement. We will send this to you directly after reviewing your product, use case, and Yelp Fusion Enterprise eligibility.

If we don’t hear back from you by 4:00pm EST on 7/23/2024, we will temporarily disable your API key until we receive a response with the above requested information.

Please disregard this email if you are already a Yelp data licensee. If you are a license holder, please send us a copy of your agreement so that we can ensure there is no disruption to your API key.

Thanks, The Yelp Fusion team

There are a lot of problems with this email. I’ll just name a few. First of all, I was not “trialing” the Yelp API. I had been using it for a decade and had official permission from Yelp to create Restaurants. Second, Yelp already had the information they requested. I sent it to them when I first created the app and requested API access. I also had a profile on the Yelp developer portal that described the app and they could easily access the Restaurants website.

Third, it was surprising to hear my usage was, “higher than lots of other Yelp Fusion developers…” That implies I am somehow abusing the system. In fact, it was Yelp themselves who (as mentioned earlier did it without my specific prompting) provided me a 25,000 per day API call limit. But the actual usage was on average below 100 API calls per day. As I mentioned, it was not that popular an app.

But the biggest problem with the email is the threat. It said that in 4-days they would shutdown my API key with no prior notice unless I responded with the information they requested. They sent the email on a Friday and wanted a response by Monday. Imagine I was on vacation? That would effectively shutdown all of my users.

I thought this must be some kind of mistake. I was not even aware that Yelp had gone to an exclusively paid-model for the Yelp Fusion API. I had not received any prior emails to that effect, and had not been keeping up with the Yelp Engineering Blog. So, the 4-day threat seemed completely out of the blue.

I responded to the email which was sent directly from a Yelp employee who I’ve chosen to remove the name of from this post and her email CCed another Yelp employee. So despite being written to “API User” it seemed like it was sent by hand and customized for each recipient.

At the time that I wrote my reply I still was not aware (based on the first email) that no free version of the Yelp Fusion API any longer existed since the email only mentioned “enterprise” and “data.” I should have checked the Yelp developer website to better understand the situation before replying but I was surprised and given the 4-day deadline wanted to get a response as quickly as possible given the threat. I wrote:

Hi -name of person who emailed me-,

This all feels a bit out of the blue with not a lot of warning. We have shared customers who have been using Restaurants for 10 years. The tone of the email feels pretty threatening after a 10 year partnership. We do not use Yelp Fusion beyond the limits that were provided to us by Yelp when the product first launched.

I asked for permission from Yelp before launching the product and received it. It mainly drives traffic to your website since you only provide three reviews and three photos per restaurant. Why would you want to shut down a product that simply serves as another way to drive traffic to Yelp? The reason you approved it 10 years ago is because you don’t make a native Mac app yourself.

The information about Restaurants you asked for can be found on its website: https://oaksnow.com/restaurants/

Please let me know what your intentions are going forward.

Thanks, David

Of course it was naive (or you could say hopeful) of me to think they would care about my little Mac app. Here’s the reply I got a couple hours later:

HI David,

Thanks for the response and information! We have converted to a paid licensing program. You will need to obtain a license with us in order to continue your usage of the API.

Without a license in place, your API key will be disabled next week. This will just disintegrate all Yelp data on your platform. You can always sign up for a free trial on our website if you need more time.

If you are interested in a license, please review the attached program and pricing overviews.

Thanks, -name of person who emailed me-

It was then I checked the website and found there was no longer a free API. I immediately removed the app from sale on the Mac App Store because I didn’t know yet if it would make financial sense to pay for the API. I didn’t want users buying the app and being shutdown next week.

Development Ends

I did email back and forth a little bit more just to say I wouldn’t have minded them going paid if I had received several months of notice, but that the 4-day notice and the threatening zero-context email was quite rude. I did this just to provide feedback since I knew the decision was fait accompli. Of course that didn’t go anywhere.

I started looking into my options. With the few sales Restaurants garners, it really didn’t make sense to start paying for the API. Especially with a company that would be willing to pull-the-rug from under me with 4-days notice. So, the app stayed off the Mac App Store. And today, 10-days after the original email, Yelp shutdown its API key.

This distresses me because there are users who purchased Restaurants (when it was on sale for $1.99 during an indie app sale) as recently as earlier this month and now it doesn’t work. Not to mention the customers who have been using it for years. But due to the way that the Mac App Store works we don’t have our customers’ email nor any way to directly refund them. So, instead I need to direct them to Apple to request a refund. If you purchased Restaurants, please ask Apple for a refund by following the directions here. Even though it was beyond my control and I took the app down as soon as I could, I sincerely regret your customer experience.

I’ve also considered moving to a different API but unfortunately no API is quite as comprehensive for restaurants as the Yelp API. Google Places is close. But it would also require a lot of re-development which is not really something I’m up for given the app’s low sales.

Lessons Learned

I have learned a lot from this experience. Perhaps the number one thing I re-learned, since it has bitten me before (hello Facebook’s Parse), is that if you utilize a third-party API for the core of your app, you are at their whim. But using APIs for many apps is inevitable. So, you need to be sure the companies you work with are not the kind that will give you 4-days notice.

The other thing this taught me is the danger of an up-front paid model for apps that depend on ongoing access to third-party services. If users were continually paying for the app, paying for Yelp’s APIs would not be as much of an issue. And I wouldn’t feel as guilty about the app being discontinued since if the fees were charged on a monthly basis, they would just end at the same time the app ceased to exist instead of facing an expectation upon purchase of “forever access.” On the other hand, how would you charge a monthly fee for an app that people are only willing to spend less than $5 for upfront?

Adding a contact to the user’s address book on Apple platforms using the prescribed method requires jumping through a lot of hoops. You have to:

• add NSContactsUsageDescription to your info.plist
• add an entitlement to your app
• ask the user for authorization (requires an in-app popup)
• if you want to modify the note field of the contact, you need to get permission from Apple using an online form to be able to add a special com.apple.developer.contacts.notes entitlement to your app; Apple may take up to two weeks to respond just to reject you

For one-off contact additions, there’s a simpler way that requires none of that. The idea is basic: you create a contact, save it in vCard format, and then ask the operating system to open the vCard file in the Contacts app. When the Contacts app opens, it will ask the user if they really want to add the contact. This requires no entitlements (even if you’re using the note field), no authorization, and even works in a sandboxed app.

The Code

I put a Swift macOS example app on GitHub. The code should be easily modifiable to work on iOS. Conveniently, the Contacts framework includes a method for serializing contacts into vCard format. You start by creating a CNMutableContact and filling it with your arbitrary data:

let contact = CNMutableContact()

contact.contactType = .organization
contact.organizationName = "Silly Restaurant"

contact.phoneNumbers = [CNLabeledValue(
        label:CNLabelPhoneNumberMain,
        value:CNPhoneNumber(stringValue:"5555555555"))]

let address = CNMutablePostalAddress()
address.street = "31 Silly Way"
address.city = "Silly Town"
address.state = "Vermont"
address.postalCode = "05401"
address.country = "USA"
contact.postalAddresses = [CNLabeledValue(label:CNLabelWork, value:address)]
contact.urlAddresses = [CNLabeledValue(label: "Apple", value: "https://www.apple.com/")]

Then you can serialize the contact using CNContactVCardSerialization and save it to a temporary file:

let data = try? CNContactVCardSerialization.data(with: [contact])
let fileURL = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent("contact.vcf")
try? data?.write(to: fileURL)

Finally, it’s just a matter of using NSWorkspace to open it in the Contacts app:

if let contactsURL = NSWorkspace.shared.urlForApplication(withBundleIdentifier: "com.apple.AddressBook") {
    NSWorkspace.shared.open([fileURL], withApplicationAt: contactsURL, configuration: NSWorkspace.OpenConfiguration()) {
        (app, error) in
        if let error = error {
            print("Had a problem trying to open the Contacts app.")
            // your error handling here!
        }
    }
}

I hardcoded com.apple.AddressBook as the app to open the vCard here, which is Apple’s Contacts app. You should probably not specify the exact address book app incase the user has a different default.

It works, but you might say, what about that note field? If you add a note property to your CNMutableContact you will notice it is silently dropped when the contact is added to the address book. This has nothing to do with the note special entitlement. It turns out CNContactVCardSerialization does not have support for either images or the note field. You can easily add both of these back. A Stack Overflow post provides some code showing how to do so.

Security Loophole?

When working on a new version of my macOS app Restaurants, I came across the note field entitlement requirement. I submitted a request to Apple using their online form to have access to the entitlement and a week later I was rejected for my request being too vague. Fair enough, it’s their sandbox, and they have the right to reject me for being too vague. But waiting so long to get an answer was frustrating and adding contacts requires a lot of ceremony. Frustrated, I went down the road of this alternative method for adding contacts.

It seems strange to me that you need authorization through Apple’s prescribed method just to add contacts but there is this “side way” using a vCard file that doesn’t require any authorization or entitlements. I’m not saying it’s a security vulnerability. It’s more of a loophole. If Apple really want developers to have to be authorized to do anything with Contacts, this shouldn’t exist.

I filed a security report with Apple thinking maybe they had looked over something so obvious. But I didn’t get a bounty. Instead, they closed it, and marked it as “expected behavior.” I basically expected as much. It seems like too obvious a thing to really be an oversight, but perhaps adding contacts should not require so many hoops to jump through using Apple’s way.

However, a key difference between the two ways of adding contacts, is that an authorized app has access to the contacts database, whereas this method using a temporary file simply opens the Contacts app, which is then in full control of the situation. It’s a lot safer. Any risk is mitigated by the Contacts app. But from a user perspective they look quite similar. In the authorization case, the user gets a pop-up within your app. In the example above, the user gets a pop-up within the Contacts app. The downside, is it does take the user out of your app. But if this is a rarely used feature of your app, maybe it’s worth it, to not have to deal with the frustration!

Although it came out in the year 2000, Weaving the Web: The Original Design and Ultimate Destiny of the World Wide Web is as relevant today as it was when it was published. Weaving the Web is a memoir by Tim Berners-Lee about the creation and early growth of the Web. Most of the book concentrates on the ideas, insights, software, and previous attempts that led to the Web—as well as the decisions and evangelism that allowed it flourish in the ’90s. The last quarter of the book deals with Berners-Lee’s ideas about how the Web should evolve. Some of the philosophy behind the Web explained in Weaving is very relevant to current debates around censorship, centralized control of content, and privacy.

We cannot fully understand something if we do not understand its origin story. Weaving the Web fills in many blanks for the curious reader. Like any great creation, the Web did not form in a vacuum. It was the result of over a decade of experimenting by its creator. In the early chapters of Weaving the Web you feel like you are there with Berners-Lee and his colleague Robert Cailliau as they pushed the Web forward. Like Jony Ive has said—ideas are fragile when they’re first created. You get a strong sense of how Berners-Lee nurtured his idea.

Perhaps even more interesting than the origin story, is the philosophy and core ideas that Berners-Lee imbued the Web with. Some of his outlook is surprising and insightful. For example, he does not credit HTTP or HTML as the most important innovation, but instead the URI. “It is the most fundamental innovation of the Web, because it is the one specification that every Web program, client or server, anywhere uses when any link is followed.” (page 39)

Throughout the book, Berners-Lee advocates for decentralization and for empowering individuals as creators. It’s important to note that the book came out at the height of the Web 1.0 era, before the onslaught of social media and YouTube-like content sharing sites. A time when the Web was very static. Yet, he didn’t intend it that way. “I never intended HTML source code (the stuff with the angle brackets) to be seen by users. A browser/editor would let a user simply view or edit the language of a page of hypertext, as if he were using a word processor.” (page 42)

The first web browser that Berners-Lee developed was also an editor. He continually encouraged companies to come out with combined browsers/editors but most declined. It’s interesting to think how differently the Web would have evolved had the browser/editor concept taken off.

Berners-Lee’s philosophy goes beyond technology. He designed the Web to be an open, decentralized system that anyone could participate in. “Whether inspired by free-market desires or humanistic ideals, we all felt that control was the wrong perspective. I made it clear that I had designed the Web so there should be no centralized place where someone would have to ‘register’ a new server, or get approval of its contents. Anybody could build a server and put anything on it.” (page 99)

As the Web has become more and more dominated by a few large tech companies, many feel this early philosophy has been lost. It’s not the current ethos. It’s not the way that most people interact with the Web. Berners-Lee was very prescient in understanding this threat. “If a company claims to give access to the world of information, then presents a filtered view, the Web loses its credibility. That is why hardware, software, and transmission companies must remain unbiased toward content. I would like to keep the conduit separate from the content.” (page 132)

The last quarter of Weaving the Web deals with Berners-Lee’s vision for how the Web should evolve. Much of it did not come to pass—at least not in the way he advocated. It includes explanations of standards like SMIL that never really took off. It speaks to how creating a standard is not as important as making a killer app. This section is interesting from a historical standpoint—to understand what people were thinking about after the first decade of the Web. But it’s not nearly as interesting as the rest of the book.

Overall, Weaving the Web does a great job recounting the story of the Web’s creation. It’s well written and insightful. Most importantly, it clearly states the philosophical underpinnings that inspired Berners-Lee and propelled the Web through its critical first phase of growth. It provides a lot of historical context and insight for many of our current debates around the Web.

GitHub Copilot and other machine learning based programming assistants will fundamentally change how we assess competency in introductory programming courses. Computer science educators can no longer rely on formulaic assignments and an honor code to ensure original work that demonstrates programmatic ability. Copilot-like assistants have blurred the line between the programmer’s work and machine generated code (largely modifications of pattern-matched work from their training sets). While they provide a productivity boost for the professional programmer and the advanced student, they may potentially act as a crutch for introductory students, who will rely on these tools in lieu of developing a strong understanding of their own.

There are certain standard problems that we are accustomed to assigning because completing them demonstrates the ability to implement fundamental simple algorithms. A prior strategy to reduce plagiarism has been to provide scaffolding code or put a spin on a problem to make it unique. Unfortunately, Copilot-like assistants are almost as capable in these scenarios as they are at writing generic simple algorithms. In my own preliminary testing on a (what I believe to be) unique scaffolded assignment of my own creation for an introductory class, Copilot was able to contextualize the comments and write most of the smaller functions accurately with just a little bit of my assistance.

What can we do about this? How can we assess competency in this environment? Over the past few decades, computer science education, like many other fields, has been moving away from exams and towards project-based learning. This has been a positive trend for a host of well-researched reasons that are spelled out in the literature. Unfortunately, I think this trend will need to be at least partially reversed for introductory courses. Students must demonstrate the ability to write and comprehend fundamental algorithmic code without the assistance of an AI. We could try banning it, but that never works well. Instead, we can try to assess knowledge “live.” How much do you know in this moment without someone or something’s assistance? And that’s what an exam evaluates.

Of course, exams have well-documented downsides, including but not limited to the fact that a significant number of bright students do poorly on them who do fine with project-based learning. I am not suggesting we return to a world of exams making up the majority of the grade in a course. Instead, I am suggesting that exams and exam-like evaluation will need to be a greater percentage of the mix. We can be creative. An oral presentation can in some instances demonstrate knowledge as well as an exam. A project coded live in class on machines that do not have AI assistants enabled, can serve as a pseudo exam.

We do not need to return to the dark ages. But we must acknowledge that these new tools mean that how we evaluate introductory programming knowledge has to change. I have had graduates obtaining first jobs in the insurance, defense, and healthcare industries building programs that have life-and-death consequences. They need to have a firm grasp of the fundamentals. What happens when Copilot makes a fundamental mistake and the person using it does not have the skill to realize?

I just had the chance to experience 8 different technical interview processes. And they were not what I expected! I didn’t get asked a single data structures and algorithms question. No leetcode for me. This may largely be related to the types of organizations that I applied. Despite writing a book about algorithms and teaching a class called “Data Structures & Algorithms,” I don’t do particularly well at white-boarding. I hadn’t done a software development job search in 10 years, and based on what I had read, and what I had heard from my students, I expected to do some white-boarding. But I didn’t have to do any white-boarding!

I wanted to write this post as a kind of follow-up to a podcast episode we did a few months ago about technical interviews. We really emphasized the prevalence of algorithmic problem solving questions and brain teasers. Maybe we got that wrong. Maybe that’s an outdated view. Or a view limited to only certain kinds of companies.

In this post, I’ll tell you about what the processes were like at the anonymized companies that I applied, how the rounds were ordered, and how the search ended for me. But first some background for context: Earlier in my career I worked as a full-stack web developer at a couple startups and then worked for a few years as a freelance iOS developer. Then I shifted to computer science education. The past six years I’ve been working as an assistant professor of CS at a teaching college. In the past three years I’ve been the co-program director of the CS program.

So, I’m a strange candidate. I didn’t know if I should be applying for mid-level roles or senior-roles. I don’t have any corporate software development experience. I do some open source work and write books and teach. I have a few very small indie apps. And as I mentioned, I worked for a few years in startups and as a freelancer. So, it was hard to tell how companies would see me.

I did a mini-job search this month. I had the option of a promotion and a new contract at the teaching college I work at, but I wanted to see what else was out there before I signed the dotted line. I want to emphasize that I was sincerely interested in exploring changing careers back into the world of software development. I was a serious candidate. This wasn’t a lark. At the same time, I knew there was a reasonably high probability I would choose to stay in my current profession.

That’s why I decided to do a very targeted search. I only applied to a select group of (mostly remote) roles that seemed like a very good fit. And I did all of the applications in 1 day. After that day, I didn’t apply to any more companies. The entire process from doing applications to offer was about 2 weeks.

How it Started

On May 2nd, I applied to a little more than a dozen jobs on LinkedIn and via the iOS Dev Jobs mailing list. I also posted on the monthly Hacker News “Who wants to be hired?” thread. That was it. I never applied to any more jobs after May 2nd.

By Wednesday I had received a few immediate rejections, 5 first-round interviews as a result of my applications, and 3 inbound first-round interviews from companies that saw me on Hacker News.

How it Was Going

I got a second round with just about every company I applied to. Apparently I did very well in those first round interviews. By the end of the first week I was very confident.

How it Ended

I’m going to break your suspense before I get into the details because it will help frame some of my decisions around why I pulled out of several of the companies’ processes. I received two offers by May 17th. They were time sensitive. I ended up accepting an improved offer to stay at my current employer. The other offer I received was for a senior software engineer position with a salary that is literally more than double my current salary. It was very compelling and I really liked the person I would’ve been working for and his team, but I ultimately decided I didn’t want to give up the academic lifestyle. I really do like where I work. It’s a great place. But it was a hard decision. I also received two outright rejections from companies that I had gone through three rounds with. I pulled out of the other interview processes after accepting the offer to stay.

The Processes

I am anonymizing all of the companies here as much as possible. I’m not here to criticize anyone specifically, but instead praise and criticize some of their interview processes more generally.

Company 1 — to be a Mac software developer

Company 1 is a small/medium sized maker of software for the Mac. I make indie Mac apps myself and first learned to program for the Mac when I was 14 years old. So, this was a very interesting company to me. And something I knew I would enjoy doing.

Application or Reach Out

I applied.

The Process

The first round was a 30 minute screen with the co-founders of the company. The second round was a technical interview that was just a conversation about some of my past work and programming philosophy with the CTO. The third round was a take-home software development project that you had unlimited time to complete and were paid to do.

Result

I pulled out before completing the third round because I had received the other time sensitive offers. I think I really would’ve enjoyed working for them, so I asked the CTO to keep me in mind for part-time or contract roles in the future.

My Feelings

This was a very good process for a small to medium sized company. It was very personable (meeting with the owners right off the bat) and yet also seemed to hit at the technical in exactly the right way. The technical conversation was non-intimidating and I think really did give the CTO a good sense of me and what my skills are. The take-home project was very relevant to the work they do and a good test, although I pulled out before completing it.

Company 2 - senior software engineer working in Python

Company 2 is a Fortune 200 company looking for an addition to a relatively small inner team that works on B2B products for an exciting and growing market in green energy.

Application or Reach Out

The hiring manager reached out to me based on my Hacker News post.

The Process

The first round was a 1 hour conversation with the hiring manager. I appreciated that he took the time to talk to me for that long instead of sending me to HR. He was erudite and personable. He was specifically looking for someone who could be an internal teacher which I really appreciated and made the job even more compelling to me. There was a short call with HR after. The real second round was a 5 hour interview with 5 inner rounds: 3 technical interviews (data modeling in Python, testing/QA in Python, and commenting on a Python pull request), a culture interview with 2 engineers, and a final interview with the hiring manager.

Result

I received a very compelling offer for more than double my current salary. I did seriously consider taking it, but ultimately accepted the improved offer from my current employer for the reasons I mentioned above.

My Feelings

The hiring manager was fantastic and I really enjoyed talking with him. If he hadn’t been so great, I could see a one hour first round being a bit much. On the other hand, the entire interview process really only had two rounds, so the entire process is very reasonable. While I don’t love live coding, the engineers on the live coding technical interview rounds really made me feel like it was a collaborative process. They definitely weren’t checking for syntax. They were checking for thinking. They helped me when I got stuck without giving too much away. The pull request technical interview was great because it really let you shine as a developer in a different way—as someone with judgement instead of just someone who can code. Overall it was one of the best renditions of live coding I’ve seen.

Company 3 - iOS developer at a consulting company

Company 3 is a reasonably well known consulting company. I used to work as a solo iOS consultant (freelancer) so this role seemed particularly compelling to me.

Application or Reach Out

I applied.

The Process

The first round was a 1 hour conversation with a senior engineer there who seemed to be guiding the process. The second round was a 1 hour technical interview with another engineer there that asked me technical questions about Swift (i.e. how do closures work in Swift). The third round was first some simple live coding in Swift, and then building a very simple table view based app in Xcode. I believe I completed all of the tasks successfully. The two technical rounds both ended with some more “soft questions.” In the third round I was asked by one of them if I had any “enterprise development experience.” And I said I didn’t have any, which is true and was evident from my resume.

Result

I found out just after I had received the two offers mentioned above that I was not selected to continue to the fourth and presumably final round. I had actually already accepted one of the offers, so I was about to pull out of the process anyway, but they emailed me first, so I unfortunately got to feel the rejection! They said the reason was that I didn’t have enough enterprise software development experience. Which makes sense because I don’t have any. But they knew that before the first round, so it was a strange reason to reject me. Perhaps they just had a better candidate who did, or perhaps they didn’t want to tell me the real reason.

My Feelings

The first round was great. They said they were interested in my book writing and podcasting work. I found the second round interviewer hard to read, but the questions he asked about Swift I thought were fair. I heard after by email that I did well. The third round was not great. It was live coding doing some fairly trivial stuff, but the lag of using a remote system did not work well for laying things out in Interface Builder. I did have trouble remembering during the live session how to do the syntax of some simple exception handling in Swift, so maybe that’s the real reason I failed that they didn’t want to tell me. I did get it right eventually, but it took me longer than it should have to remember the syntax. However, I don’t think asking live syntax questions is a good way of conducting technical interviews—in the real world we can just look it up. And I did politely criticize the laggy setup at the end too by suggesting it would’ve made more sense to let me share my screen instead of remotely controlling theirs (probably not a good move to criticize your interview setup). Ultimately the feedback for why I didn’t get to the final round felt too short—I’d like more detail. And if the “not having enterprise software development experience” was the real reason they should have screened me out after the first round, not the third. I think doing a live coding session in Xcode makes little sense in the iOS world—let us do a take home or look at our prior open source work or indie apps. That’s a better test of what we can do. However, I sincerely liked the company and could see myself applying again if I ever do another job search.

Company 4 - senior iOS developer at well known iOS app company

Company 4 is very well known. Their iOS app has millions of users. They were looking for someone to be in a senior role on the iOS team. They described it as a leadership role in the interviews.

Application or Reach Out

I applied.

The Process

The first round was a 30 minute screener with HR. The second round was a 30 minute technical conversation with the head of the iOS team. For the third round you were offered either a 2 hour live Swift coding test or a take home. I chose the take home. It was building a fairly simple iOS app but with plenty of room for embellishments. They said you had as much time as you wanted, but also said not to spend more than a few hours on it. After two days I got a message from HR saying they were eagerly awaiting my submission (I hadn’t started because of other interviews and them saying I had as much time as I needed). I told them that I had been busy (the truth) and would finish it by the end of the weekend (4 days later, which I did). I sincerely thought I did a fine job on the app. I even added a couple extra features, like some custom bar chart drawing, that was not required in the app description.

Result

I was not selected after the third round. The email from HR said “Thank you for taking time out of your schedule to complete our assignment. We appreciate the time and care that went into completing the assignment. At this time, we have received several assignments back that have set our candidate caliber very high. It has not been an easy decision but we will not be advancing to the next round interview. Please continue to view our job board for any future openings you are interested in.”

I replied because I really wanted more specifics after doing the take home because I thought it was reasonably good: “Thank you. I understand and appreciate the consideration. If I could receive any feedback about what I could have done better on the assignment I would appreciate it so I can improve my work in the future.”

The final reply from HR was: “The feedback was all positive, we just didn’t have a good match on the technical side. I do wish you the very best.” I’m not sure exactly what that means. Was the feedback all positive on the technical side but the techniques I used were not a good match? Was my code amateurish in style but good logically? Was my use of older technologies not good? I really would’ve liked more feedback after the time spent on the take home.

My Feelings

I felt this company had a very fair process. But their final feedback was not great at all. After spending several hours on an unpaid take home project, you’d like to at least know the specifics of why it wasn’t good enough or why overall you weren’t a good candidate. That would have to come from engineering, not HR of course. But HR could pass it on. The most likely explanation is that they were telling the truth in the first email—there simply were higher caliber candidates applying at the same time. But the second email then made me doubt that. What does it mean for the feedback to be all positive but to not have a “good match on the technical side.” What is it that I could’ve done differently to bring my take home to the next level and be a good technical match? I sincerely wanted to know. I thought my app was pretty good for the time they told us we were expected to put into it (“a few hours”) and not getting to the final round honestly gave me a bit of imposter syndrome :).

Company 5 - educational non-profit lead developer position

This is a non-profit with a strong engineering culture. They were looking for someone to lead one of their large initiatives which is based on a Web app written in Ruby on Rails.

Application or Reach Out

I applied.

The Process

I first had a 15 minute chat with the founder. He was super nice. The real first round though was a 1.5 hour conversation with him. It was wide ranging. We talked about technical stuff including him asking me to show him some of my work, and he asked me some well prepared interview questions about product improvements they could make and how I saw the product. The third round would’ve been working directly on their codebase (paid) to implement a couple features after speaking with two of their engineers for about an hour. But I pulled out before the third round. One thing I really didn’t like was that in preparation for the third round you had to sign up for all of their services (Slack, Stripe, Logging service, etc.). In total I received 7 invite emails for the third round. It was a bit much.

Result

I pulled out of this one before even receiving the other offers because I had a bad feeling before the third round. I decided I really didn’t want to work in Ruby on Rails (bad memories of a startup I did 10 years ago) and I also had a feeling that I wasn’t a good culture fit. I felt a bit bad pulling out the night before the third round but I didn’t want to waste their time or my own.

My Feelings

It’s a super cool non-profit but the third round process didn’t make a lot of sense. Sure, you want someone to show they can work on your actual codebase. But requiring them to sign up for 7 different services is a bit much when they are likely doing other interviews. I’m glad I pulled out because I did a lot of hard thinking about it and really knew it wasn’t a good fit for me technically or culturally. It was a tough decision though because they really do have a great mission and great founder. But I think this made me realize that the technology and culture I’m going to be working in is as important as the product.

Company 6 - developer relations at advanced startup

This is an exciting startup that’s pretty far along. They have a product that is solving real problems for real clients already. Developer relations seemed like a good intersection for me between my technical communications skills (teaching, writing, podcasting) and software development abilities. In some ways it seemed ideal, but at the same time I didn’t have any background in working with their type of product.

Application or Reach Out

They reached out to me based on my Hacker News post.

The Process

The first round was a 30 minute screen with one of the founders. The second round was a 45 minute conversation with their head of engineering. The third round would’ve been a written interview (you fill out 5 detailed question in a Google Doc).

Result

I pulled out after the second round because I received the other offers and had to decide between them. The timeline just didn’t work out. But they actually reached out to me after I pulled out to ask if I would reconsider. That was super unexpected and flattering. I was very impressed by them overall.

My Feelings

It was clear in both interviews that the team is experienced and intelligent. I have regrets about not getting to work with such a sharp team in a role (developer relations) that I think really exploits all of my different skillsets. At the same time, I was a bit hesitant to work in developer relations on a product that’s in a space I didn’t know anything about. I have a strong feeling they will be very successful and kind’ve have some FOMO about it. I also regret not getting to do the written interview process because I pulled out. I’ve never done a written interview before and it seems like a very unbiased way to judge a candidate.

Company 7 - iOS lead at small startup

This is a startup in a space that I would call a form of social networking. They were well funded, but funded by friends & family, which is a bit of a red flag for me based on my earlier career.

Application or Reach Out

They reached out to me based on my Hacker News post.

The Process

The first round was a relatively casual chat for over an hour with one of the founders. It was really wide ranging and I felt like we really connected. The next round was supposed to be a conversation with the more technical co-founder, but it never happened before I had to pull out due to the other offers.

Result

I pulled out as mentioned above because of the other offers.

My Feelings

This company is early enough and small enough that they didn’t have a great technical screening process in place. I actually mentioned that to the founder and maybe it’s something I’ll even help him with in the future. We’re still in touch by email and the product is definitely interesting. At the same time, I would’ve had some real hesitancy to join a startup funded by friends and family because I did that a decade ago and we ran out of funding.

Company 8 - software developer at huge multi-national

This was a poorly defined role at a subdivision of one of the most well known technology companies of the 20th century. The job is at an office near where I live, so I figured it was worth a shot to at least find out more.

Application or Reach Out

I applied.

The Process

The HR representative reached out to me by email. I replied. She didn’t. She reached out to me again by phone. I replied. She didn’t. We finally connected for the first round, which was an HR screen.

Result

I pulled out during the HR screen. The salary was too low.

My Feelings

This company is out of touch and it showed even in the application process. The UI of their job site was the worst I’d seen. Also, companies should post salaries on job descriptions so as not to waste people’s time.

Remainder

I got rejected without even an HR screen from about seven other companies that I applied. They were mostly big companies. This kind of makes sense because my resume is very non-traditional for these kind of roles and I doubt someone who is looking at me as the whole person and potential developer (for example looking at my open source projects) is necessarily working in HR at a big company. Smaller companies seem to more often have people with software development experience doing first rounds.

Conclusion

Overall, I got a great offer in just two weeks. I can’t really complain about that. And interestingly it was from someone who reached out to me on Hacker News instead of somewhere that I applied. While I ultimately just accepted an improved offer at my current employer, I came away from this process with three takeaways:

1. If you have a non-traditional, but strong background, having that be well understood by a hiring manager is really important. You need to put all of yourself out there and get that information to that hiring manager in as succinct a form as possible. My Hacker News post did that for me.
2. A lack of feedback after doing a take home test doesn’t leave a candidate feeling good. Employers, please respect that people put time into doing a take home and give them detailed feedback.
3. leetcode problems do not seem as prevalent as the zeitgeist would have you think. Or maybe that’s the selection bias of where I got interviews. I mostly didn’t interview at very large corporations.

Today I launched a new novelty Mac app, Retro Dither. Retro Dither gives any photo a cool retro look using just black and white pixels. You may want this for artistic effect, or you may want to export your photo to MacPaint for display on a retro Mac. Retro Dither launched on the Mac App Store today.

Note that we’re talking about just black and white, not grayscale (no grays here!). Amazingly, dithering algorithms make pure black and white images look like grayscale. You’ll get an old-school newspaper look, or a look like your photo just came off of a Mac from the 1980s. And this works with any photo in any bitmap format (JPEG, PNG, HEIF, etc.). Retro Dither can then export your dithered image to MacPaint format for display on black and white Macs going all the way back to the 1986 Mac Plus.

This is the story of how I developed Retro Dither…

I was working on my next programming book, which will be an intermediate Python projects book, when I came across an article about Atkinson Dithering on Hacker News by John Earnest. Dithering algorithms can be used for approximating the look of an image with less colors. Atkinson Dithering is one that is particularly well suited for approximating an image using just black and white. I was in the middle of developing a project for the book related to computational art and I was thinking I would need another art project to fill out the chapter. And Atkinson Dithering really caught my eye. First, because it looks cool. Second, because I was a Mac user growing up and I remember all the software and games that actually used it.

But there was a problem. After reading John’s article, I found the algorithm too simple to sit alongside the other projects I was developing for the book. I can explain the gist of it to you in a few sentences. You take each pixel in an image one at a time from the top left of the image to the bottom right. You check if it’s closer to black or white. Whichever it’s closer to, you turn it into (if we stopped there, we’d have a simpler dithering algorithm known as Threshold and it wouldn’t look great). Then you take the difference between black or white and the original pixel and call that the “error.” You spread that error into some of the pixels to the right and below the original pixel. Then you continue to the next pixel. John explains it in more detail in his article.

Then I got an idea. Bill Atkinson, the creator of Atkinson Dithering, was also the creator of MacPaint, a program I loved growing up. What if we not only dithered the image, but then we exported it to MacPaint format? That would be really cool and would add a bit more heft to the project. So, I started researching the MacPaint file format and found this article. It’s not a particularly complicated file format, but it at least includes another useful algorithm, Run-Length Encoding, a simple compression scheme where instead of writing out repeated sequences like “AAAAAAAAA” we essentially would write 2 bytes that say “9 As”. Now we have two simple, but interesting algorithms. That’s perfect for a project in the book.

I had to learn a bit more about the MacPaint file format. For example, it encodes pixels at 1 pixel per bit before it does the run-length encoding on the pixel-packed bytes. 0 is for white pixels and 1 is for black pixels. It runs the run-length encoding on a per-scanline basis. Every MacPaint file contains exactly 720 scanlines and 576 pixels per scanline (basically a 576 x 720 resolution). I found my hex editor, Hex Fiend, really helpful when debugging. I would look at my output MacPaint files and compare them to example MacPaint files, including some I still have saved from my childhood. I also found an open source project that converts files the other way (MacPaint -> modern bitmap). I could run my program’s converted MacPaint files back through it and see if the same image came out the other end.

With the help of Pillow for reading the initial bitmap image for conversion, I was able to build the whole thing in a couple hundred lines of Python. Again, this was the perfect size project for the book, so I was feeling really good about it. Then I tried my MacPaint files in actual MacPaint and there was a problem. They wouldn’t open!

Thinking back to my early days in computing, I remembered about type and creator codes. These were metadata that the Classic Mac OS (versions 1 through 9) used to associate a file with the program that should open it instead of using a file extension. The problem was my files had none. I used a Classic Mac program to add them back in and my images successfully displayed in MacPaint! But this wasn’t an ideal workflow. I wanted my users to be able to just double-click their finished files. So, I wanted to add the type and creator codes into the exported MacPaint files from my Python program.

The problem was, the Classic Mac OS stored files in two separate sections, known as data forks and resource forks. It stored the type and creator codes in the resource fork. Almost every other operating system, including modern macOS, Linux, and Windows doesn’t have resource forks. Files are just a single blob. And my research was showing me that there would be no easy way to add a resource fork to my file for transport so that the Classic Mac OS would see it as one file. I needed a work around.

Of course tons of people had the same problem before. Classic Mac OS users had to work with people on other operating systems. So transport file formats were developed that could combine data forks with resource forks together in one file for transport and then “unbundle” them on Classic Mac OS so they could be used. The simplest and most common (other than the later developed .sit, Stuffit archives, basically Zip files in Mac land), was MacBinary.

Turning my MacPaint files into MacBinary files for transport with correct type and creator codes was as simple as adding a 128 byte header to the file in MacBinary’s specified format. This is still not a perfect solution, because the files need to be unstuffed by a program on the Classic Mac OS side that understands MacBinary before the MacPaint files can be double-clicked, but programs that understand MacBinary were common on the Classic Mac OS and include the aforementioned and ubiquitous Stuffit.

I got this all working in a couple nights for the book and then I thought, surely someone else must make a commercial MacPaint converter. The retro Mac community is pretty huge right now. But I couldn’t find any, other than the venerable Graphic Converter, which is a full-featured graphics program that I recommend (its creator, Thorsten Lemke, was actually gracious enough to let me conduct an interview with him for a research project in college back in 2006) but that costs $30.

I decided to make something lighter weight. Thus, Retro Dither was born. I ported my Python code to Swift. I added 4 more dithering algorithms (they’re all similar to implement). I built an AppKit frontend. I tested it. I tested the output files in emulators like SheepShaver for MacPaint 2 on Mac OS 9 and Mini vMac for MacPaint 1.5 on System 6. I did find a couple bugs, like that MacBinary expects each fork to be in exactly 128 byte chunks (I just had to fill extra space with some 0s). Even though later versions of Stuffit will ignore this, other MacBinary unbundlers like the built-in MacBinary command-line app on modern macOS and the original MacBinary program from the the standards authors in the 1980s will not (by the way, the modern macOS’s Archive Utility will unbundle MacBinary files).

I hope you enjoy Retro Dither. And if you want to be made aware of my next Python book when it comes out, which will include the project described here, please join my very low volume mailing list.

The common criticism of the modern news industry is that objective journalism has been replaced by opinion-slanted proselytizing. There’s an interesting slice of the news world where proselytizing was never even at issue. I’m talking about the world of news sites, blogs, magazines, and rumor mills that cover Apple.

I’ve been reading news sites like MacNN (now defunct), MacWorld, and MacRumors daily for more than two decades. I also enjoy more personal “news” and commentary blogs like Daring Fireball and Six Colors. And then there’s the journalists like Mark Gurman and David Pogue that at some point in their careers covered tech in an Apple-centric way for real news organizations like Bloomberg and the New York Times. There’s a cottage industry of journalists and outlets that make their entire living on the happenings of Apple Inc. Surprisingly, many of the specialized sites even managed to scrape by when Apple was merely a ten billion dollar company, and not the two trillion dollar behemoth it is today.

There’s an important assumption about the readers of these outlets. It’s assumed you’re a fan of the company. Why would you read MacWorld or a lot of what David Pogue wrote back in the heyday of his Apple fervor if you didn’t own an Apple product? But being a fan of Apple generally means more than being a user. In the 1990s when, as a kid, I first became a rabid Apple fan, it was known as a cult (in fact one of the news sites is still known as Cult of Mac). Why were you using the non-standard products of this small struggling computer company with less than 5% marketshare? You had to believe in their products’ superiority or the potential of the company’s future despite the clear market signals to the contrary. You had to believe. It was a religion for techies.

That belief in Apple and its products has always been imbued in the outlets that cover the company. The fans demanded it. It was why we kept reading. We wanted our faith to be reinforced despite all of the contrary voices. Reading the news sites was like going to church. It made sense in the context of the ’90s. They were the underdog. It was fun and cool to root for the rebels. And the potential was there. If we could just keep them going with our collective enthusiasm, they could reach their true destiny.

But then they did. They became the largest company in the world. They came to dominate several slices of the tech industry. You’d think that would mean the coverage of them would change. Yet, the cottage industry that covered them never lost its fervor. It’s like the prophecy was fulfilled and now those who believed would go to heaven and be angels, rewarded for their faith.

And angels they are. Towards the company. The new presumption is no longer that some unrealized dream could be achieved when the right products arrive. The new presumption is that the company is a force for good. How did this transformation happen? It used to be about “could they deliver the long promised tech future.” It was about Mac vs Windows. For a while it was about iOS vs Android. Now it’s about social issues, battles with government, and court cases.

I think that we’re all looking for something to believe in. And many of us, including the “journalists” and commentators that cover Apple, are not finding it in the rest of our lives. Yet somehow our belief in this company during its dark times, against all odds, was right. And so we want to continue to believe our faith in it will be right again. Yet, today, we’re being asked to believe in something very different. A different company facing a different set of challenges in completely different spheres. I’m still a big fan, but I no longer take every leap of faith.

I’ve played far too many hours of Civilization throughout my life. If you haven’t, or you’re not an aspiring game designer, then you may find large sections of Sid Meier’s Memoir! hard to appreciate. But I do love Sid Meier’s games and I do dabble in making games as a hobby. So, I really appreciated this memoir that takes you through Meier’s entire career as well as some of his rules of game design that have made him so successful.

This book is not a deep dive. Each chapter, which generally covers a game or two, provides some insight about how a game was developed and why it worked (or did not work) for the player. But the chapters are short, and many of them concentrate on just one aspect of the game or Meier’s philosophy. The two things I found most interesting were the inspiration behind each game, and the gameplay rules that Meier learned over time.

It’s not a tell-all, and it doesn’t cover much about the video game business. You won’t get a play-by-play about the development of Civilization VI. Meier says that he left much of the business side to his partners. Instead, this is a book about the creative process.

Meier touches on enough of his personal life to give you a general outline of his life and some of the real-life hobbies that led to some of his best games. Again, most of the details are left out, but there’s enough there to be inspiring. The thing I found the most surprising in the entire book, is that despite his high position, Meier continued to be an active participant in the programming of his games (or at least the prototypes) late into his career.

In short, Sid Meier’s Memoir! is limited in its scope. It’s mainly about game design and the creative process. But it does that well. You won’t quite feel like you’re in the trenches with the team making Civilization, but you will feel like you have a sense of where their head was at. The lessons about game design and the creative process that Meier provides are drawn from deep experience and most seem self-evidently true. The writing is clear and the insights are real. If you loved his games or you want to make games, you should read this book.

Instagram is one of the most important cultural phenomenons of the 2010s. No Filter by Sarah Frier covers Instagram’s rise from a two-person startup to the second largest social network in the world. Along the way, Frier explores Instagram influencers, the network’s effect on teens, and its sometimes rocky existance within the confines of its parent company, Facebook. No Filter is competently written and it seems Frier had significant access to the important players. Yet, the narrative lacks a compelling spark, and despite the access, fails to reveal much insight beyond what is already well known about the company.

No Filter is written in the “anonymous amalgamation of interviews” style of other recent hit business books like Hatching Twitter and Console Wars. This means that the interview source of each bit of original reporting in the book is not directly revealed in a footnote or endnote. This is not only problematic for the book as a historical document, it also means the reader is not aware of who’s bias is tilting a particular anecdote. This style can be partially justified if it leads to a more compelling narrative. However, while I found Hatching Twitter and Console Wars to be page turners, I didn’t feel the same way about No Filter. It may just be that Instagram’s corporate history is not particularly exciting compared to Twitter or Sega, but I also think it had to do with the way the book is structured.

The narrative is somewhat abruptly interrupted halfway through to spend a couple chapters on Instagram’s cultural effects, and the people who make their living through the app. Although I am not a regular user of Instagram, I was already aware of basically all of the material in these chapters just from loosely following the news and the cultural zeitgeist surrounding the company. I didn’t feel these chapters added enough value to be worth the interjection during what should have been the heart of the book.

Like many business books written by journalists about software companies, No Filter’s coverage of the technical aspects of Instagram is inadequate. Instagram co-founder/CTO Mike Krieger’s technical leadership is praised, but the reader does not get a sense of what kinds of technical decisions he really made that were successful. What technical hurdles did Instagram face as it scaled? How did they solve them? This is ultimately the story of a piece of software and part of its success is due to the engineering of that software. That angle deserved better coverage.

Speaking of Mike Krieger, we don’t get to know him very well in No Filter. In fact, the only people we get a strong sense of, are Instagram co-founder/CEO Kevin Systrom and Facebook CEO Mark Zuckerberg. It’s like a book about the early days of Apple that talks about Steve Jobs, but only scratches the surface with regards to Steve Wozniak. Okay, Mike Kriger might not quite be a Steve Wozniak, but he probably deserved a few more pages.

Frier does a very good job with Systrom, though. The reader gets a strong sense of his personality. Frier plays up the battles and contrasts between Systrom and Zuckerberg very well. We come to understand that some of the decisions made at Facebook surrounding Instagram were personal, and others were based on the very different characters running the two social networks under the same roof.

When you add up the “influencer” chapters and the lack of character development for many of the players, No Filter starts to feel like a relatively short book. Yes, Instagram is not a very old social network, but given the author’s access, there was surely more to be revealed. I wanted more business, technical, and character narrative and less filler.

Ultimately, No Filter is not a bad book. If you want a cohesive 300 pages about Instagram, you will get it. It’s just a bit of a bland book. It may be that Instagram’s corporate history is not that exciting. Or, it may be that the author missed some opportunities. Either way, I commend Frier for taking on Facebook, a powerful entity in our society that deserves further examination. And her portrayal of the relationship between Kevin Systrom and Mark Zuckerberg was very well done.

Surprisingly, for a memoir about a guy who spends most of his time sitting in front of a computer, Ghost in the Wires: My Adventures as the World’s Most Wanted Hacker is a gripping thrill ride. And it’s a testament to Kevin Mitnick and his co-author, William Simon, that it works. In the wrong hands, the same material could easily be boring or overly technical. They found the right balance of action, technical detail, and non-hacking content to keep the story engaging. The book gives you a real sense of the hacking underworld, as well as how surprisingly easy it is to social engineer individuals, companies, and government agencies.

The most surprising part of the story is how little of Mitnick’s exploits were due to what we traditionally think of as computer hacking, and how much was due to social engineering. Mitnick learned to be a master manipulator, and in this book he explains to you how he convinced cops to turn over records, trusted employees to send files to people they just met, and just about anyone to do anything over the phone. If you want to better protect yourself from social engineering, this book is a great primer. It really makes you think about how quick we are to trust someone with a bit of confidence when they know a couple details we assume they would only know if the confidence were warranted.

Mitnick manages to make himself relatable. By including personal details, descriptions of family life, and imagery of his surroundings, he comes across as a regular guy. He compares his hacking activities to an addiction. I can almost buy that. He was getting notoriety, solving interesting puzzles, and probably feeling the same kind of rush that cleptomaniacs feel. It sounds like it could easily become a compulsion if you’re good at it and don’t have a certain moral wavelength turned on.

And I say that last sentence carefully. Because Mitnick (as far as I know and he claims in the book) never did anything especially damaging compared to some of the other well known hackers. He says he wasn’t in it for money or to do harm, but instead to satiate his curiosity. A significant portion of the book concentrates on this fact, and how the media played up his story to make him sound a lot more evil than he deserved. And that apparently affected his prosecution by the government. John Markoff, a well known New York Times reporter at the time, is singled out for particularly incredulous stories.

Yet, my biggest criticism with the book, would be Mitnick’s lack of sympathy for his victims. He spends a lot of time emphasizing how little damage he did, and almost no time apologizing for the damage that he did do. Sure, he may not have sold the source code he stole for a profit. And sure, the people he tricked mostly just had their time wasted. He didn’t actively try to ruin anybody’s life. Yet, mitigating his “work” inevitably cost companies and individual a lot of time. Employee time is money. It probably cost taxpayers millions of dollars investigating, trying him, and catching him. His exploits made people feel unsafe and caused them emotional distress. And he doesn’t seem very sorry. For that reason, I found him especially difficult to root for during the early chapters. Even his “pranks” as a teenager sometimes seemed mean spirited if I were to be on the receiving end of the frustration they caused.

In the end, though, Mitnick won me over. I found his relationship with his mother and grandmother endearing. I think the way he turned his life around after getting out of prison the last time is remarkable. It seems he’s done a lot of good the last twenty years. He’s an example of why people deserve a second chance, and his book is an interesting examination of social engineering and the media-legal system complex.

A reader recently complained to me about the hyperlinks on this blog. The reader thought the links were too hard to distinguish from the rest of the text. And the reader’s right. The Swedish Greys desktop theme that I thought looked cool eight years ago, while attractive in an aesthetic sense (to me at least), is not the most usable or accessible. I’ll be looking for another theme.

I was able to style my blog however I wanted to and it looks the same in all browsers. That’s the flexibility of good HTML/CSS standards. Every site can look and behave exactly as the creator envisioned. It’s also why the Web’s a usability nightmare. We have to learn to use every site we visit because every site is designed differently. How come when we talk about Web standards the focus is almost entirely on technical standards? Where is the worry about design standards?

I recently finished reading the classic book The Design of Everyday Things by Don Norman and it talks about standards. Normans says “When all else fails, standardize.” Basically when you have no other way of implementing good design, you turn to standardization so at least every user only needs to learn how to use the similar things (in this case web pages) once. And I think we have no other way, because if we did, we would have figured it out in the past 30 years.

It wasn’t always this way. I remember using the pre-CSS and pre-JavaScript Web as a little kid on Mosaic. You knew there that the blue underlined text was always a hyperlink. And you knew that the back button always took you back a page. And you didn’t have to worry what different actions buttons did, because there was no JavaScript. I’m not saying we should go back there, but in many ways having the constraints made pages easier to use. There was no need to think. Now we have no constraints, but that’s why we need standards.

Every other major consumer computing platform but the Web has design standards. Apple’s platforms are famous for their Human Interface Guidelines. They are an attempt to ensure all apps follow some standard design conventions. Not every app does, but Apple has some ability to enforce them through its app stores, and some users even demand developers follow them. So, they are at least kinda sorta followed by most major apps. If the Web had design standards, maybe users would demand developers follow them too. Google and Microsoft have design suggestions and guidelines for their developers. This is why a good app for each platform feels “at home.”

But we have no design guidelines for the Web that are widely accepted. Sure, people have tried. But the only way we’re going to get something that’s actually followed is if we have a standard. And a standard needs to come from a standards body (Apple, Microsoft, and Google are the standards bodies for their respective platforms). W3C, please put some focus on a design standard. Not everybody will be forced to follow it, but it could do a lot of good in terms of usability.

I am pleased to announce that my fourth book, Classic Computer Science Problems in Java, has been published. It is now available for purchase from the publisher’s website. As Manning’s deal of the day, the book is available today (January 5, 2021) for 50% off.

Classic Computer Science Problems in Java is a continuation of the Classic Computer Science Problems series, with previous incarnations in Swift and Python. They teach problem solving techniques from the realm of computer science in an approachable code-centric tutorial-like fashion. They are relatively light on theory and heavier on analogies, examples, and code. You don’t need a computer science education to pick up the books. In fact they were designed with self-taught programmers in mind. You do need to be at least an intermediate programmer. You can find out more about the contents of the series at classicproblems.com.

The Java book follows the success of Classic Computer Science Problems in Python which has sold more than ten thousand English copies and has been translated into eight other human languages including Portuguese, Simplified Chinese, German, Russian, Polish, Korean, Traditional Chinese, and Japanese. Beyond the three original book programming languages, the source code has been ported by the community into five additional languages including Go, C++, Ruby, PHP, and JavaScript.

I don’t know if the Java book will be as successful as the Python book, but I do know that there will be a lot less readers upset about the inclusion of type annotations. Manning provides a short free sample of the book on their website, so you can check it out before you buy. Let me know if you have any questions on Twitter. I’m @davekopec.

I give a lot of presentations. As a college instructor, I present at least once every week, and usually multiple times. I’ve also presented in business contexts and to general audiences. The Presentation Secrets of Steve Jobs: How to Be Insanely Great in Front of Any Audience by Carmine Gallo caused me to rethink how I’m delivering my content. I had seen every Steve Jobs video available online, including all of his keynotes, before reading the book. But it took Gallo’s insights to distill what made Jobs’s presentations so great in a digestible form that I could use.

Gallo’s book is based on the premise that Steve Jobs’s presentations were excellent. If you don’t agree with that and you didn’t like his style, then don’t pick up this book. You can watch any of Jobs’s presentations on YouTube. And if you haven’t, I highly recommend watching some of the most famous ones (introduction of the iPhone for example) before picking up the book. They will give you context because Gallo refers to specific presentations in just about every chapter. In fact, if you haven’t seen them, you may want to watch them as they’re introduced in the book, although it will add many hours to your reading experience.

Most of Jobs’s presentations featured slides. Therefore a significant amount of the advice in the book relates to making slides. If your presentations don’t involve slides, you will find about one third of the book’s content not applicable to you. Jobs’s slides were very different from most of the slides we see in business and academia. They had almost no text. Gallo clearly explains how Jobs made this work

Much of the non-slide advice is common sense: practice, vary your use of voice, be passionate when presenting, use demos, break up your presentation, etc. Yet the combination of all of it into a single volume and the way that Gallo elucidates these points make the book a one stop shop for improving your slides. I never got bored reading this common sense because I love Steve Jobs and even though I already knew almost all of the anecdotes in this book, they really made the advice come alive. If you don’t know much about Steve Jobs then you might not find the book quite as exciting.

The Presentation Secrets of Steve Jobs is a great book to improve your presenting skills, especially if you make presentations with slides. Its advice about making slides is unconventional and will make you rethink how you do them. Its other advice is mostly common sense but is really well illustrated. If you’re not familiar with Steve Jobs or you have not watched his keynotes you should watch some of them either prior to reading the book or simultaneously with when they are presented in each chapter. If you don’t like Steve Jobs/Apple then this is not a book you will enjoy.

Thomas Alva Edison is almost inarguably the greatest inventor of all time. The first device for recording and playing back sound, the first large scale power distribution system, the first practical light bulb, and the first commercially successful video recording and playback device were just a few of his numerous inventions. His inventions span the gamut from mining, to electricity, to chemistry, to telecommunications, to optics, to mechanics, and even to botany. Edmund Morris’s Edison is an excellent book by a very experienced biographer that takes you through every one of Edison’s major inventions with enough context to help you understand how and why they came about. But Morris made a very strange structural decision that, while interesting, takes away from the book’s enjoyment.

Morris’s biography dispels two myths that I had heard about Edison (having only previously read a dime store biography of him and a book about his movie production company). The first is that Edison doesn’t deserve credit for his inventions because it was his workers who really came up with most of them. It’s clear from this meticulously sourced biography that Edison was not only the visionary behind his inventions, but that he also had a thorough scientific knowledge that allowed him to work hand-in-hand with his workers, directing them at every step. This is all the more amazing since he had no formal education and was thoroughly a self-made person. Edison was incredibly studious and consumed volumes and volumes of both scientific and non-scientific literature. Most of his major inventions really were his ideas. Some of them were improvements on the work of other inventors, and some of them were co-invented with his workforce, but he was the indispensable figure who created the first industrial research laboratory, making this all possible and directing it meticulously from his own mind.

The second myth I had heard about Edison is that he was a good scientist but a terrible businessman. It depends what you mean by “businessman.” Morris makes it clear that Edison was poor at managing his finances. But what is also clear from Morris’s account is that Edison was an amazing entrepreneur. He was able to see what inventions had practical commercial potential. He turned ideas into money. Did he then subsequently run those businesses well? Not generally. He was a great entrepreneur, but not a great manager.

Beyond invention, Edison’s life was filled with serendipity, interesting personal stories, and meetings with the greatest figures of his generation. Morris spends about three fourths of the biography on Edison’s inventions and business pursuits, and the remainder on the rest. He was perhaps the most famous person alive during his lifetime, and for good reason—his inventions utterly changed the every day lives of the masses. Reading the biography, you will get a sense of this, but it’s not delved into in any great detail. The majority of the pages are spent on the inventions.

The explanations of the inventions are clear enough that you do not need a scientific background to understand the gist of their purpose. You won’t understand them at a fundamental level, but you will understand why they were important and how they connected to the rest of the Gilded Age world. There’s not room, even in eight hundred pages, to go into all of Edison’s inventions, nor all of his personal life. His journey was just that amazing. But you get a sense that Morris chose the important highlights well.

So, Edison is a well-written and balanced biography about one of the most interesting figures of all time. What is not to like? The structure. Morris chose to bizarrely present Edison’s life in reverse chronological order. Each chapter covers a decade of his life. And the book starts with his last decade, and ends with his first. As we all know, lifetimes progress linearly. We’re used to hearing stories in order. It’s hard to keep track of characters, when we are abruptly and repeatedly thrown back ten years in their lives. The result is that it’s very hard to keep straight the lives of Edison’s associates and family members as you read Edison. It’s interesting in the last two chapters, knowing what you know about his life, to hear about how it started. It’s interesting, but I don’t think the trade off was worth making the rest of the book harder to read and keep track of. Morris died about five months before Edison was published. I wonder if he had lived longer, if he would have rethought the structure upon further reflection. Some reviewers have suggested reading the book backwards. I don’t think it would hurt you much to do this.

Thomas Edison was an amazing, interesting, and world-changing figure who deserves a thoroughly researched extensive biography by a great writer. In Edison, we have that. Its bizarre structure notwithstanding, if you are at all interested in technology, business, or even just history, you should give Edison a read.

No Rules Rules: Netflix and the Culture of Reinvention is not a history of Netflix. It’s an extended account of the corporate values that have resulted in Netflix’s success as told by its cofounder/CEO Reed Hastings and accomplished business writer & academic Erin Meyer. This well written and clearly explained book offers insight into managing a creative company using non-traditional management techniques that give employees greater freedom and responsibility. However, the techniques are likely not as widely applicable as the authors imply.

No Rules Rules follows a unique format, in which each author’s voice is clearly pointed out in their sections of each chapter, leading to a kind of dialogue between the two. This format creates balance. Each point is discussed from the insider/pragmatic perspective of Hastings and the more academic perspective of Meyer. Meyer will sometimes backup Hastings’s assertions with research outside of Netflix, or gently pushback against some of his more absolutist tendencies. Meyer appears to have had significant access to employees throughout Netflix while doing her research. However, there’s no section in which she completely disagrees with Hastings, and throughout most of the book the reader is simply getting the same point from multiple perspectives.

The book revolves around the benefits of a corporate culture that empowers individual contributors to make decisions without bureaucratic tape and draconian oversight. This is meant to increase efficiency, improve flexibility, and help employees feel more satisfied with their roles. A couple specific examples are employees deciding for themselves the appropriate amount of vacation to take each year (no vacation policy) and signing contracts without getting approval from their managers. To get to this place of what the book calls “freedom and responsibility” there are certain prerequisites defined by the authors. These include a culture of candid feedback and achieving a high “talent density.”

These corporate values have obviously served Netflix well, but they may not quite be the panacea they seem from reading the book. Unfortunately, despite Meyer’s involvement, the values are somewhat myopically presented within the confines of Netflix. Probably the most controversial point in the book is its assertion that “adequate” employees (sometimes interchangeably referred to as “good” employees) should be let go to make room for hiring “great” employees. This is presented rather uncritically, without the obvious introspection that it is easy for an industry leading organization like Netflix to have its pick of “great” employees waiting at the gates to get an opportunity to work for it when they let go of the “good” employees.

The authors only caveat is that safety or process oriented companies (think nuclear reactor or industrial manufacturing) cannot risk freedom and responsibility. Yet, there are many other creative types of companies that cannot fulfill the prerequisites outlined by No Rules Rules. For example, there are creative companies that due to the limited profits in their industry cannot pay top-dollar and cannot risk letting go of “good” employees because there will be no “great” employees waiting to take their place.

No Rules Rules is a good book for learning more about Netflix, how its corporate culture works, and how it has helped to make it successful. It’s sprinkled with enough interesting anecdotes and good writing to keep your attention. The unique format adds value and the non-traditional management techniques are interesting and surely have some merit. Yet, the book’s failure to acknowledge the unique circumstances of Netflix that make its unique culture possible, stop it from being a “great” book. It’s just a “good” book.

I am what I would call a non-famous public person. What do I mean by that? I mean that I have jobs where people publicly review my work. As a college instructor, I receive student reviews. Some of them get read internally at our school, and some of them get posted publicy. And they’re attached to my name. They’re not some product where people might just know my company or my brand. They know me, and they’re reviewing me. As an author, my books get reviewed. Some of those reviews are by editors and official reviewers internal to the publisher. Some of those reviews are by email. But most of those reviews are posted online. As a software developer and hobbyist podcaster I also get public reviews.

To anyone in a personalized creative field who cares deeply about their work, bad reviews hurt. It’s not just about being sensitive (although I am sensitive and that doesn’t help). It’s that when you really pour your heart into something, as I do my teaching for example, then the product is an extension of you. Nobody would deny that the way a class or a book turns out is an extension of the personality, knowledge, and ability of the teacher/author. And when the product is criticized, it’s like you are being criticized.

My dad, who was also a college instructor, and the producer of nine books and nine instructional videos among many other public creative ventures, had a very thin skin. He couldn’t stand me reading him a bad review. That stuck with me. This very accomplished man, couldn’t stand even a little criticism. When I get too sensitive, I try to remember how his inability to accept criticism hurt some of his products. He was much more about getting things done than doing them perfectly. If he had a little bit more of a perfectionist streak in him, he would have done even better. But maybe he would have done less…

It doesn’t matter if I receive eighteen great reviews for a class. It’s the words of the two bad reviews that stick with me. And do you know why? Because almost always there’s a kernel of truth to those reviews. Does that mean we should listen to them? Well if I only listened to the bad reviews, I would never produce anything. I would just stay in bed. The fact is the people who are getting no bad reviews, are the people who are not taking the risk of putting themselves out there. So, we have to not let ourselves get so discouraged that we stop producing. And we have to remember that while we may not have served those two students well, we may have served those other eighteen students better than they would have been served in a world without us teaching them.

We can’t please every student or every customer. Nothing that gets watched/read by more than a few people is going to be liked by all of them. Even Mother Theresa would get a downvote on YouTube. So, if we’re serving the majority, we don’t want to let the critiques of a few steer us in the wrong direction.

On the other hand, being like my dad is dangerous too. No, the customer is not always right. But usually the customer has a point. We have to always be working to get better. Because we’re not perfect, and there’s always room for improvement. And we need to read those bad reviews to find those spaces for improvement. It doesn’t mean they’re right, but it does mean we need to think about them.

My dad accomplished an amazing amount. If he let those bad reviews slow him down too much, it’s very possible he would have not accomplished as much. But if he could have found a few tweaks, a few changes, to serve a few customers better, well then he could have grown through them. I want to grow, but I also don’t want to stop. So, I can’t let the reviews get me too down, but I have to read them.

A child refugee builds the first wholly female software company, putting feminism into practice, and goes on to become one of the most noted philanthropists in her country. It’s a story fit for Hollywood. Stephanie Shirley tells it with gusto, raw honesty, and compelling writing in her autobiography, Let It Go.

As a half-Jewish child during the beginning of World War 2, Shirley was saved from the horrors of Nazism by escaping on a train as a child refugee. In England she would build a new life, and build a software consulting company. Freelance Programmers (later F International) employed female programmers who were under-appreciated and discriminated against by the business world at the time. Many of these women worked from home part-time or while raising children in a world that wouldn’t allow them to work otherwise. They would work on large technical projects for some of the largest companies in Britain. Let It Go is a strong feminist treatise drawn from reality, not abstractions.

Shirley was a pioneer in many ways. Creating a software company in and of itself was still a new idea in 1962 when she got started. Creating a company of freelancers decades before “the gig economy” was an innovation. Employing almost all women in a technology company (although women were many of the first programmers) was unheard of. In Let It Go Shirley takes you through the early struggles of building Freelance Programmers. The reader receives a strong understanding of how the company operated, how it acquired clients, and the challenges that it faced as it grew from her cottage into a multi-million dollar enterprise. While Shirley covers the early period in the right level of detail, the company’s later growth is not as well recounted in the book.

Let It Go is also a deeply personal book. Shirley goes into detail about her relationships, her family life, and her struggles taking care of her child, Giles, who is affected by a severe form of autism. She weaves the personal stories well into the business narrative, making it a more “full picture” story than most business books. Her personal life affected her business and vice versa. That’s true of almost anybody, and to think otherwise is a folly. Yet many business memoirs almost completely exclude personal details.

One of the things I appreciate most about Let It Go is that Shirley held nothing back. Even embarrassing or difficult details were not left out. If she felt jealousy or betrayal over a colleague’s actions, she explained it. She covered her mental breakdown. She covered marriage difficulties. She covered her fraught relationship with her parents. It was real. It was raw. And it made for a more realistic, interesting, and insightful journey.

The latter half of the book deals primarily with three topics: Shirley’s transformation of her company into a partially employee-owned operation, her struggles to help her son with his condition, and her philanthropy. Shirley ended up being a leading figure in philanthropy in Britain, and in autism philanthropy in particular. She founded a school for children with autism, the country’s largest autism charity, and too many other philanthropic initiatives to mention here.

The theme of the book can be seen as “relinquishing control,” hence the title Let It Go. Shirley ultimately let go of her personal tragedies, her business, and many of her philanthropic ventures (to other managers). And she sees them as more successful as a result of it. It’s a powerful message that can only be fully understood by reading the book. If you have any interest in the early software consulting business, the evolution of autism care, women working in technology, or just want to read a great life story, Let It Go is worth your time.

Blood, Sweat, and Pixels: The Triumphant, Turbulent Stories Behind How Video Games Are Made by Jason Schreier is a compilation of vignettes about the development of ten different video games. Many of the games are RPGs, and most of them are produced by large studios. Despite the many different accounts, there is a real lack of diversity in the stories. Almost all of the stories deal with the same themes: crunch (working large amounts of overtime to ship a game), overcoming mismanagement from above, and reworking stories/gameplay that are found in development to be lackluster. The stories can therefore seem repetitive, despite being generally well written with a great degree of access to the original developers. The vignettes are too short to let the reader dive into the finer aspects of each game’s development, and it is hard to feel connected to the many introduced characters.

Although many of the ten stories feel repetitive, there are a few gems that stand out. The story of Eric Barone developing Stardew Valley by himself over five years was perhaps the most interesting, and is almost worth picking up the book for alone. It deserved more than just one chapter. Because it focused on just one person, it was able to better weave together the game and the personal story. Which really highlights the problem with the book as a whole: it tries to cover too many stories, and therefore ends up giving every story too little space. Schreier seems to have had excellent access to the developers, but this access was squandered by being so scattered.

Another strong chapter in the book is on the development of The Witcher 3. Schreier covered the interesting corporate history of developer CD Projekt, who’s unique foundation and Polish roots made for some less humdrum reading than many of the other large studio stories. The story of Yacht Club Games and Shovel Knight was the lone other indie chapter beyond Stardew Valley. Like Stardew Valley it was one of the more interesting chapters because it had fewer characters to keep track of. The book would have been better if it were ten chapters on five games, instead of ten chapters on ten games.

There is also a distinct lack of focus on overcoming technical hurdles and the software development/programming that goes into making games. The book focuses a lot more on game design and creative decisions, rather than technical work. Aspiring game programmers may come away disappointed. And by leaving this huge aspect of game development largely out of the book (there are some chapters that mention it, but without any level of detail), I feel the author did not fully represent the development process. On the other hand, not getting too technical probably made the book more appealing to a broad audience.

Blood, Sweat, and Pixels is an ambitious book that falls short of its lofty goal. It is probably best read piecemeal, picking and choosing the stories that interest you most. However, the author deserves credit for taking on such a challenging task, getting real access to the pertinent parties, and giving the reader an inside look at game development despite some poor editorial/structural decisions.

I am pleased to announce the availability of my fourth book, Classic Computer Science Problems in Java. You can now purchase early access to the book from Manning. Use promo code ccspkopec for 40% off. This is the third book in the Classic Computer Science Problems series, following Classic Computer Science Problems in Swift and Classic Computer Science Problems in Python. You can find out more about the series at classicproblems.com. The manuscript is complete but it is being rolled out in chunks for reader feedback over the summer. Hopefully, the final version will be in print by the end of the year.

The books are aimed at intermediate programmers who want to delve deeper into the covered problem solving techniques, brush up on core algorithms, or learn more Swift/Python/Java using problems familiar to them from other languages. They are suitable for professionals looking to deepen their understanding of the covered topics, students with some programming background looking to expand their computer science knowledge, and anyone preparing for coding interviews.

What is a “Classic Computer Science Problem?” It is a problem one typically finds in an undergraduate computer science curriculum. The topics covered in the books span the gamut from core computer science algorithms you would find in a data structures & algorithms class to artificial intelligence and its sub-discipline machine learning. There are both practical and whimsical problems. They include classic search problems, constraint satisfaction problems, graph algorithms, genetic algorithms, k-means clustering, simple neural networks, and more!

They’re meant to be approachable broad books, not deep books. They’re not textbooks. They’re hands on code-centric tutorials to pique your interest in the many topics covered.

All of the code in the most recent book is available on GitHub. The book takes advantage of features up to version 11 of the Java language. The code only uses the Java standard library throughout, so there’s no wrestling with third-party libraries. You will learn how to solve all of the problems in the book “from scratch” so that you gain a deeper understanding of how each problem solving technique works under-the-hood.

To learn more, checkout the Classic Computer Science Problems in Java page on Manning’s website where you will find a full table of contents and temporary free access to small portions of the book.

We decided to bring the book to Java (a language close to my heart because I first learned it as a teenager over 20 years ago and have worked professionally in it), because the Python version has been a huge success. It has sold many thousands of copies and has been translated into eight human languages in addition to the original English. The Swift version did only moderately well, so we are excited to bring the book to another mainstream language audience. We try to be careful to make sure each version uses best practices in its respective language, but I would love to hear your feedback about the Java version during the early access period for any ways we can improve on our use of the Java language.

Remember that you are purchasing a pre-release version of the book, so you will be joining me on the journey to its final release in the fall. You will be receiving rough drafts of chapters before they have been fully vetted. I encourage you to send me your feedback, but keep-in mind that these are early days and everything is not yet perfect. You will receive the final version of the book upon publication.

The Declaration of Independence is considered to be one of the most important documents in human history. Not only was it the seminal document in the political formation of the United States, it also was arguably the first time that a nation was expressly formulated with an understanding of the essential nature of human rights, despite not living up to it. It inspired many more rebellions against tyranny around the world in places as far removed as Vietnam and Haiti, and continues to inform our understanding of the relationship between a people and their government.

However, while its syntax is beautiful, it also no longer rings true to everyone who reads it. This is both because of our updated understanding of who political rights should be applied to (all human beings), and because not every reader is familiar with the assumptions and meanings inherent in the eighteenth century English of its writers. I don’t think we should literally update a historical document, but I do think it’s important to be clear about what the Declaration means to us today.

Here is the language in the Declaration’s most famous section:

We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness.–That to secure these rights, Governments are instituted among Men, deriving their just powers from the consent of the governed, –That whenever any Form of Government becomes destructive of these ends, it is the Right of the People to alter or to abolish it, and to institute new Government, laying its foundation on such principles and organizing its powers in such form, as to them shall seem most likely to effect their Safety and Happiness. Prudence, indeed, will dictate that Governments long established should not be changed for light and transient causes; and accordingly all experience hath shewn, that mankind are more disposed to suffer, while evils are sufferable, than to right themselves by abolishing the forms to which they are accustomed. But when a long train of abuses and usurpations, pursuing invariably the same Object evinces a design to reduce them under absolute Despotism, it is their right, it is their duty, to throw off such Government, and to provide new Guards for their future security.

Here is my interpretation of the Declaration’s most famous section in the simplest language that can represent what it means to me in today’s context, and no simpler:

We believe that all people should be treated equally under the law. There are universal human rights including, but not limited to, the rights to life, liberty, and the pursuit of happiness. Government exists to ensure these rights. Government’s only legitimate power comes from the people it governs. Governments that infringe on human rights or derive their power from a source other than the people, are illegitimate. It is the right and duty of people to replace such governments.

This is my personal interpretation of the Declaration. This is what it means to me today. And it’s how I will parse it for my children. Maybe I have some details wrong and I’ve changed some of its intent. But I think the spirit is right and I think the language will be very easy for them to understand and contextualize for the modern world.

One year ago, I launched BTV Daily, a daily email newsletter that delivers news, events, top social media posts, and other items of local interest to subscribers in the Burlington, Vermont area. The newsletter is largely automated, although I write a short daily blurb in a section called “Dave’s Corner.” BTV Daily is a hobby, but one that I take quite seriously because it provides real value to its subscribers. I know this because for the one-year launch anniversary this week, I put out a subscriber survey, and I was impressed to find how many people really appreciate the newsletter.

This post contains details about how I built BTV Daily and how I’ve grown it from 0 subscribers to (a still fairly meager) 296 subscribers (as of today). Keep in mind that Burlington, Vermont is a city of just 40,000 people. Also, keep in mind that the newsletter is not (yet?) monetized. In fact, it costs me money. This is a passion project, not a business.

Generating Content

To see what a copy of the newsletter looks like, you can checkout the archive section of the BTV Daily website. The sections in the newsletter are Weather, This Day in Vermont History, News, Events, Top Twitter Posts, Dave’s Corner, Latest SeeClickFix Issues, and Quote of the Day.

Technical Setup

BTV Daily is generated by a Python script that runs via a cron job on a Raspberry Pi in my home. At 7:45 AM each day, the Pi connects to GitHub to pull my latest blurb for Dave’s Corner. At 8 AM each day, the main script executes. It calls various APIs to generate BTV Daily’s content, which is amalgamated into an HTML file that is passed to MailChimp to deliver to our subscribers. I use no images in the newsletter to decrease load time and avoid being marked as spam. I do make extensive use of emoji instead. Unfortunately, I still get erroneously marked as spam by gmail’s spam filter for some users despite the fact that this is a subscribe-only newsletter (I don’t manually add any users).

Weather

The day’s weather information is generated using the Dark Sky API. An avid reader asked me to include sunset time in it, and I think that was a very nice addition. Apple has purchased Dark Sky, but their API will keep working through the end of 2021.

This Day in Vermont History

I added this section about halfway through the year after receiving permission from the Vermont Division for Historic Preservation to use their database (which they sent me as a Word document). This was, of course, very nice of them. I wrote a script to reformat the database into a Python dictionary.

News

When I first started BTV Daily, I used the RSS feed of The Burlington Free Press to generate the news content. However, much of the content from them is paywalled, and users complained. So, I moved to the Bing News API, which Microsoft provides a Python library for accessing. I carefully calibrated the local sources to request stories from, the keywords to use to get Burlington specific stories, and other settings. I also added the NY Times as a source, so that when Burlington is in the national news, relevant stories will come up. This did lead to some Bernie Sanders political stories making their way into the newsletter even though they were not really about Burlington (sometimes stories about him will mention his hometown of Burlington in passing).

Events

The newsletter links to Burlington area events from Seven Days, hopefully driving some traffic to this venerable publication.

Top Twitter Posts

I use the Twitter API to search for the top five tweets in the past 24 hours that have the most favorites that use hashtag #BTV or #BurlingtonVT. #BTV was unfortunately getting me tweets from non-Burlington sources, but I later discovered that the API has an option to limit searched tweets to a radius around a zip code. I use a 250 mile radius of Burlington and it works well.

Dave’s Corner

I write Dave’s Corner in a text file that is labeled by date in a format the script looks for. I can include HTML tags. I should probably put the work in to include markdown support. Occasionally, my wife will write a blurb and the section changes from Dave’s Corner to Rebecca’s Corner if the first line is “Rebecca.” My blurbs are usually observations about Burlington or my life.

SeeClickFix Issues

SeeClickFix is a tool for citizens to report local issues to municipal authorities. I have some concerns about the fact that people can be virtually anonymous on the site (they can act almost as a secret police). By exposing issues to the general populace through the newsletter, hopefully they are getting more scrupulous attention. I report the SeeClickFix issues of the last 24 hours via the SeeClickFix API. They are released under a non-commercial license. If I monetize the newsletter in the future, I suppose I may need to remove this section, or is it fair use?

Other APIs

I use the MailChimp API for actually sending the newsletter. I use the WikiQuotes API for generating a random local quote at the end of the newsletter. Unfortunately, the only non-political and non-controversial figures from Burlington on Wikiquotes are Ethan Allen and John Dewey. These quotes get pretty boring, pretty quickly.

Getting Subscribers

I got my first twenty subscribers through a Reddit post. I got another twenty or so subscribers by mentioning the newsletter on a Vermont email social network called Front Porch Forum. Several people in the one-year anniversary survey said I should post more about the newsletter on Front Porch Forum, but believe it or not the advertising rates on Front Porch Forum are exorbitant. And I don’t want to post regularly there without paying for advertising since that doesn’t feel right.

The rest of the ~300 subscribers have come through Bing ads, Facebook ads, Twitter ads, and word of mouth. I also tried a few physical advertisements on bulletin boards around the city. It ends up costing me about $1/subscriber when using Facebook/Twitter ads. I guess that’s my “customer acquisition cost.” Since this is a hobby project that I enjoy, I don’t mind that I’ve spent a few hundred dollars on it.

Keeping Subscribers

The vast majority of people who subscribe to BTV Daily stay subscribed. The numbers show that over the first year, about 80% of people who subscribe don’t unsubscribe. I try to be responsive to individual emails I get from subscribers. Several improvements to the newsletter have been a result of reader feedback. About 40–50% of subscribers open any given edition of the newsletter within 24 hours.

The vast majority of subscribers live in Burlington or one of the surrounding communities. Once in a while someone from out of state will accidentally subscribe and they will realize their mistake and unsubscribe. The most infuriating thing is when someone unsubscribes after being on the list for months and marks the reason to Mailchimp as “spam.” This happens rarely, but how can a newsletter that you personally signed up for and read for months be spam?

Looking Forward

I would like to add additional sections to BTV Daily. One section in particular that I am interested in adding is job postings in the Burlington area. I contacted Craigslist legal for permission two times, one year apart, and nobody got back to me. I applied to become an Indeed.com publisher almost a month ago and never heard back as well.

I would like to grow the newsletter to 1,000 subscribers (2.5% of the population of Burlington proper). I am thinking of showing a progress bar in the newsletter. Perhaps that would encourage subscribers to tell their friends. Or perhaps it would discourage them because they thought the newsletter had thousands of subscribers and it only has ~300.

If I reach the 1,000 subscriber goal, I may try to monetize the newsletter to at least cover its advertising costs by allowing people to place local ads. I would not be opposed to making a profit either! But it’s not my primary concern right now.

If you have questions or comments about the newsletter, feel free to reach out to me at newsletter at btvdaily dot com

I enjoy books about tech history and business. I also enjoy biographies. So, Showstopper!: The Breakneck Race to Create Windows NT and the Next Generation at Microsoft by G. Pascal Zachary was a perfect fit for me. It has a compelling software business narrative, backed up by significant author access to the major players, and features non-stop action throughout most of the book.

Showstopper, written in 1994, is a book about the building of Windows NT, one of the last still-in-use desktop operating systems to be developed from scratch (Windows NT remains the underpinnings of Windows 10). Zachary had incredible access. He was able to interview all of the major players involved in NT development, including David Cutler, the project’s lead, and Bill Gates, the CEO of Microsoft at the time. It provides real insight into the market landscape at the time, the challenges that Windows NT faced, and what is was like for regular software developers and management to laboriously crank out NT over many sleepless nights throughout a period of roughly four years.

Zachary does a good job balancing vignettes about management with vignettes covering lowly software developers, testers, and their families during development. He pays attention to the human story. What was the toll of the breakneck development schedule and the high pressure environment on families and worker mental health? He clearly did his research, took the time to interview everyone relevant that was involved, and weaved their respective narratives into a cohesive largely chronological whole.

Where Showstopper falls short is in Zachary’s understanding of the technology. While seemingly written for a mainstream audience, I imagine most readers today, like me, will be software developers. From the beginning it was clear to me that Zachary did not fully grasp all of the software development technology that a book like this inevitably needs to cover. Or if he did, he dumbed it down too much for my liking. He did his best, and I think if I were a mainstream reader, his explanations would actually be quite good: just enough to give me a basic understanding. But as a software developer, I was left wanting.

The parts of Showstopper I liked least were the first thirty pages, largely covering Cutler’s career at Digital, and the Afterword in the 2008 edition with Zachary pontificating about 2008 Microsoft. I think Showstopper was at its best when reporting on the week-by-week challenges and worker vignettes during NT development, and at its worst when trying to analyze the big picture. Another problem with the book is that it tries to cover too many characters. It was easy to lose track of who was who. You will be treated to many mini-biographies, which while interesting, are not enough to get you invested in each of the players.

Despite its flaws, Showstopper! is worth reading because it pulls back the covers of a Herculean software project in human terms. If you are interested in computer software history or the business history of Microsoft in the early 1990s, it’s a must read. Software developers with an appreciation of computer history will find it compelling and enthralling, if they make it past page thirty.

Approximately one year ago today, my most recent book came out, Classic Computer Science Problems in Python. It is the second book in the Classic Computer Science Problems series. You can purchase it on Amazon or through the publisher’s website. It has been an amazing year for the book.

I had the good fortune to be on several podcasts to promote the book. I really appreciate all of the podcast hosts who gave me the honor of being on their show. It made a huge difference in sales. For example, my appearances on Talk Python to Me and Podcast Init alone accounted for approximately 1,000 sales according to the publisher. To see all of the podcasts I’ve been on, checkout the interviews section of classicproblems.com.

This has been the most successful book that I’ve written by an order of magnitude. It’s surpassed the sales of Dart for Absolute Beginners and Classic Computer Science Problems in Swift by a multiple of their combined sales. I appreciate all of the support from the community. I think Classic Computer Science Problems in Swift was a good book but the publisher was right in saying that it needed a larger language audience. Python not only has more users in absolute numbers, it also has more users in one of the major target demographics for the series, which is self-taught programmers who missed out on a computer science education.

I’ve done many things to promote the book, and I appreciate how my publisher, Manning, has advertised the book to their audience. However, more effective than anything have been the podcast episodes. I can’t tell you how many people have told me they learned about the book through a podcast episode. Podcast episodes have even led to translations. If I hadn’t appeared on the mostly Portuguese (but sometimes English) podcast Castalio, then perhaps the Portuguese edition would not have come out.

Amazingly, the book is now on target to be translated into seven languages, including: Russian, Polish, German, Traditional Chinese, Japanese, Korean, and Portuguese. Interestingly, the only translation publisher that has been in regular contact with me has been O’Reilly Japan. I think they did a really nice job on their edition, and I thank them for their attention to detail. It’s interesting to see how the book’s teaching style (informal in terms of theory/math, to the point, and code heavy) appeals to some cultures and not to others. Anecdotally, my worst review has been for the Russian edition, and some of my best reviews have been out of Germany for the English edition. Of course I can’t tell you if the Russian translation is good or not, since I don’t speak Russian.

By far the most controversial decision I made about the book was to use type hints/type annotations throughout. This decision alone led to some of the worst reviews of the book. Would I do it again? It’s impossible to say how many people bought the book because it was one of the first Python books to use type hints throughout. I do think the type hints add value in terms of readability once you get used to them, but I can definitely see how the effect of turning some people off, makes any benefits not worth it. And of course I agree that the way Python does type hints is still too verbose.

Looking forward, one nice thing about the Python book versus the Swift book is how much more stable a language Python is. I wrote both books according to the latest versions of their languages respectively. However, by the time the Swift book came out (it took 6 months to go from done to store shelves) it was already a little outdated due to the fast pace of change in the Swift language. Python on the other hand, has a much longer shelf life and greater backwards compatibility between versions.

Finally, I am currently writing the third book in the series, Classic Computer Science Problems in Java. I look forward to it being released later this year, so look out for it if you are a Java aficionado. To decide what language to do next, we did a poll of Manning readers that had 300 votes. Java practically tied with Go. The idea of going with Java is that this book did well in a large language community (Python) and only okay in an emerging language community (Swift). Writing a book in Java feels like going back to my roots since Java was one of the first “real” languages that I learned in the late ’90s. I have done some professional projects in Java as well, so rest assured that I’m competent to write it!

I’d like to thank all of the readers who have purchased the book. I’d like to thank anyone who’s shared their constructive feedback. And, I’d like to thank the team at Manning. If you are a reader and you like the book, please do leave a review on Amazon or your place of choice.

Also, it goes without saying, but please stay home during the coronavirus crisis if you can. Stay safe!