GeistHaus
log in · sign up

x + 3

Part of xplus3.net

stories primary
In praise of middle management
Etceteraaimanagement
There's only so far you can scale with one person directing a team of juniors. Good management is our current bottleneck in scaling up AI agents.
Show full content

Inspired by a recent post from Sam Altman (2025-02-09):

We are now starting to roll out AI agents, which will eventually feel like virtual co-workers.

Let’s imagine the case of a software engineering agent, which is an agent that we expect to be particularly important. Imagine that this agent will eventually be capable of doing most things a software engineer at a top company with a few years of experience could do, for tasks up to a couple of days long. It will not have the biggest new ideas, it will require lots of human supervision and direction, and it will be great at some things but surprisingly bad at others.

Still, imagine it as a real-but-relatively-junior virtual coworker. Now imagine 1,000 of them. Or 1 million of them.

Having had some “real-but-relatively-junior” coworkers, I’m familiar with how much supervision and direction they need to contribute to a product/company/team. I invest X units of my time in them; they produce X * Y units of output. If all goes well, Y > 1; reality often falls short.

There’s only so far you can scale with one person directing a team of juniors. I can work with a few; some other engineers may be able to direct a few dozen. Eventually, you get to the point where you’re spending all of your time supervising and none of it producing that output yourself. Congratulations, you’re a manager. (Yes, this is a gross oversimplification of the manager’s role.)

Is it productive for a CEO to directly supervise and direct a thousand (or a million) junior employees? I imagine few (if any) would think so. So our CEO hires managers, who hire managers, who hire managers… who supervise and direct a reasonable handful of junior employees (often mixed in with senior employees who can take an active role in mentoring the juniors). These managers coordinate their teams with all the other teams to achieve the company’s vision, with the coordination challenges growing geometrically as the company scales.

I’ll not predict whether/when AI may eventually replace senior employees or managers. But until it does, good management is our bottleneck for scaling up our “real-but-relatively-junior” armies of virtual coworkers.

https://xplus3.net/?p=884
Extensions
European Tour
EtceteraAustriaEnglandEuropefoodFrancegalleryGermanyhistoryLiechtensteinLondonMunichParisRomeSalzburgtravel
Several years ago, Modern Tribe invited me on the annual team trip. Instead of the “usual” Central American/Caribbean destination, we were meeting in Tuscany. After the trip, Stephanie would join me in Rome, where we would have a few days to explore before heading home. This adventure was scheduled for May 2020—it didn’t happen. Jump … Continue reading "European Tour"
Show full content

Several years ago, Modern Tribe invited me on the annual team trip. Instead of the “usual” Central American/Caribbean destination, we were meeting in Tuscany. After the trip, Stephanie would join me in Rome, where we would have a few days to explore before heading home. This adventure was scheduled for May 2020—it didn’t happen.

Jump forward four years. The kids are teenagers, finances are more flexible, the world is open again. We decided it’s time to try again, this time as a big adventure for the whole family. In May 2024, we went on the Brinley Family Grand European Tour, a 17-day trip with stays in Rome, London, Paris, and Munich.

I will not herein attempt to capture the entire journey. Rather, I want to highlight a handful of experiences and create a space to share selected photographs. (Stephanie has her own selection of photos over at Laughter & Dance.)

Notable Sights
The Colosseum, which apparently had a retractable roof
Rome is full of “nasoni”, fountains dispensing potable water
Titus visited the Arch of Titus
Friendly ravens at the Tower of London
Much Ado About Nothing at the Globe
Royal Albert Hall, during setup for a concert
Musicians have to compete for space with storage carts in Westminster Abbey
Mona Lisa is hard to see
Liberty is more vibrant after restoration
At the Ministry of Justice
At the Senate
The last two standard meters in Paris
Perspectives on the Eiffel Tower
Liechtenstein might be the most beautiful place in the world
Great Food

Everything was delicious. Our first evening in Rome included a food tasting tour, and a few friends recommended restaurants. Otherwise we just searched Google Maps for nearby restaurants with 4.5+ ratings.

Ravioli at Piccolo Abruzzo
Rosso fuoco at Eggs
Pastries and chai at GAIL’s Bakery
Cream tea at The Ivy
Pasties from The Pasty Shop at the Victoria Station
Chicken cordon bleu French tacos at O’Tacos
Galette at Chez Eugène
Macarons from Pierre Hermé
Kebab bowl at La Casa de Kebab
Kalbsgeschnetzeltes at Berggasthaus Masescha
Magnificent Organs

We encountered a plethora of churches/cathedrals/chapels filled with beautiful organs, although we only had the opportunity to hear two of them: in St. Paul’s Cathedral for the Sunday eucharist, and a short concert in the Salzburg Cathedral that featured three of the seven(!) organs therein.

St. Paul’s Cathedral
The Chapel of St. Peter ad Vinicula,
Tower of London
Westminster Abbey
Royal Albert Hall
Royal Chapel, Versailles
Salzburg Cathedral
Kollegienkirche, Salzburg
Liechtensteinisches LandesMuseum
Odds & Ends
Paris
Rome
Space invader mosaics
Friendly helmets
Cat sanctuary / ancient Roman ruins
The Forbidden Lawn
Street Fighter in the Musée de l’Armée dueling exhibit
Javascript error at the Vatican museum
https://xplus3.net/?p=804
Extensions
How to reduce latency with Samsung Galaxy Buds on macOS
Etceterabluetoothmacos
I use Samsung Galaxy Buds as my primary headphones. They’re great. Sometimes, though, I notice that audio coming from my Mac is delayed by ~1 second compared to the video. After a bit of digging, I’ve discovered that macOS defaults to using the SBC codec, which does some buffering that can lead to that delay. … Continue reading "How to reduce latency with Samsung Galaxy Buds on macOS"
Show full content

I use Samsung Galaxy Buds as my primary headphones. They’re great. Sometimes, though, I notice that audio coming from my Mac is delayed by ~1 second compared to the video.

After a bit of digging, I’ve discovered that macOS defaults to using the SBC codec, which does some buffering that can lead to that delay. Switching to the AAC codec (or fiddling with SBC settings) can eliminate that delay.

The magic incantation:

sudo defaults write bluetoothaudiod "Enable AAC codec" -bool true

Reconnect your headset, and the delay is unnoticeable.

https://xplus3.net/?p=796
Organize SSH config with Includes
Codingconfigorganizationssh
My SSH config file has grown organically over the years. Overall, I try to keep it organized and well-commented. Today I thought to ask if I could include other files in my config, to separate groups of related servers into separate files. The answer, delightfully, is yes! You can include specific files, or even an … Continue reading "Organize SSH config with Includes"
Show full content

My SSH config file has grown organically over the years. Overall, I try to keep it organized and well-commented. Today I thought to ask if I could include other files in my config, to separate groups of related servers into separate files. The answer, delightfully, is yes!

You can include specific files, or even an entire directory. I went with the latter:

Include config.d/*

Adding this line to the top of ~/.ssh/config means that I can create as many files as I want in ~/.ssh/config.d/, and any Host stanzas in those files will be included as if they are in ~/.ssh/config.

Now I have a separate file for each customer who’s servers I access: ~/.ssh/config.d/customer1, ~/.ssh/config.d/customer2, etc. Any time I add a new customer’s servers, I can add a new file and it’s automatically included. This makes it easier both to see which servers are associated with a customer and to see which customers’ servers I may have access to (for when I need to clean up later).

https://xplus3.net/?p=783
Where the Sidewalk Ends
Codingperspective
Software developers work on the border between problems that have already been solved and the uncharted wilderness of possibility.
Show full content

In my work as a software developer, I solve problems. Specifically, I tend to solve problems that require me to write code that interacts with other systems, frameworks, APIs, etc., extending them to do things that they were never designed to do. Some of those systems are designed to be extensible, others… less so. And it’s typically in those hard-to-implement edge cases that my skills are most called upon.

I work on the border between problems that have already been solved and the uncharted wilderness of possibility. Living in this rustic environment, I spend most of my day staring at flaws and limitations. If only this system exposed the data I need; if only that API didn’t have a jarring inconsistency; if only I could trust the value returned by this method call. But these things exist as they are, and it’s my job to find ways around them. Despite some internal grumbling, it’s usually quite enjoyable work.

Many of the products and platforms I’m trying to extend were designed with particular audiences in mind. Customers in the target audience have common problems that can be solved with common solutions. The problems I’m trying to solve… these are not the common case. When a product successfully anticipates the needs of a customer, when it solves the customer’s problem with minimal effort, when it’s exactly the right tool for the right job… I never see these projects. They’re someone else’s responsibility, because they don’t require my particular skills.

As I poke and prod at these systems, as I uncover the edge cases that push them to their limits, I need to remind myself that my challenges and my frustrations are not indicative of the quality of the product. A product that solves most of a problem for some percentage of its audience (whether that be 95% or 5%) is to be applauded. It’s doing its job well for those users. Even if it’s not for everyone, even if it doesn’t solve every problem, for some users, this is just the solution they need. For everyone else… it’s a big world; there’s room for another product in the market.

https://xplus3.net/?p=625
Docker Desktop Filesystem Caching: Faster with Mutagen
Codingcachingdockermutagenperformance
I started down the Docker path for my local dev environment six years ago. As soon as an alpha version of Docker for Mac was available, I installed it to replace my boot2docker-based VM. I mentioned at the time that its one major drawback was performance of the osxfs filesystem. All these years later, and … Continue reading "Docker Desktop Filesystem Caching: Faster with Mutagen"
Show full content

I started down the Docker path for my local dev environment six years ago. As soon as an alpha version of Docker for Mac was available, I installed it to replace my boot2docker-based VM. I mentioned at the time that its one major drawback was performance of the osxfs filesystem. All these years later, and it’s still sluggish compared to the native filesystem.

I’ve tried various solutions to mitigate the issue. NFS volumes could give a minor performance boost for some applications, but its effects were negligible for my own dev experience. I tried docker-sync for a while, but constantly ran into problems with the sync lagging or stalling. Mutagen seemed similarly promising, but would run into the same issues.

Exciting News!

‘Twas with great delight that I read the announcement that the Docker Desktop team would finally be implementing a solution. That brings us, a couple months later, to today, where I’ve had the opportunity to test it out.

The syncing solution is built on top of Mutagen. Though I’ve had my issues with it in the past, I’m hopeful that the Docker Desktop team’s official blessing and support will help the tool become efficient, stable, and reliable. It took a little bit of troubleshooting to get to a working installation, so I thought it best to document my steps.

Configuration

To start with, the official “Edge” release of Docker Desktop for Mac is outdated (yup, that’s what I said). Instead, you can find links to newer versions in the GitHub forums. Today I’m running on build 45494, which I found in a discussion about excluding files from the sync. This build resolves two key issues that I ran into with the Edge release. First, it opens up file permissions on the synced files to resolve write permission errors. Second, it adds support for a global Mutagen config.

The Mutagen config is an essential tool for excluding certain files/directories from the sync. In my particular case, I don’t want my node_modules directories to sync. I use nvm and run my node commands on my host machine. Excluding these directories can cut a large chunk off of the synchronization time. So I created my config file at ~/.mutagen.yml with the following rules:

sync:
  defaults:
    ignore:
      vcs: true
      paths:
        - node_modules

Only after this file is in place can I configure caching according to the documentation. If you enable it beforehand, you’ll have to remove the directory from your config, restart Docker Desktop, and then re-add it.

Troubleshooting

I ran into some errors with symlinks in my project directory. Mutagen will complain and refuse to sync if there are absolute symlinks in the cached directory. Fortunately, I was able to remove them from my current projects. Otherwise, and option might have been to use the global config to ignore them.

The debugging output is not particularly helpful. When Mutagen encounters an error, all you get is an “Error” status in the File Sharing settings of Docker Desktop. Another comment in the forum showed me the proper path to viewing the error. The docker daemon’s HTTP API will show the state of the sync, along with any error messages (note that jq is here to make the output prettier).

curl -X GET --unix-socket ~/Library/Containers/com.docker.docker/Data/docker-api.sock http:/localhost/cache/state | jq

With all of my errors resolved, I can now start up my containers with the synced directories. The application performance is noticeably faster, with WordPress pages loading in a few hundred milliseconds instead of a few seconds. This shaved about 80-90% off of the total time to run my automated test suites under the osxfs mounts.

Docker Desktop file sharing settings

After a couple of days running, I haven’t seen any show-stopping issues with this new caching. Nice work, Docker Desktop team. I’m looking forward to watching this tool stabilize and improve.

Update (2020-07-07): The latest edge version of Docker Desktop makes this even simpler. By using the delegated mount strategy, Mutagen will be automatically enabled for the directory. According to the discussion, future versions will also allow one to disable the Mutagen caching for a directory by explicitly setting the consistent or cached strategy.

https://xplus3.net/?p=663
Extensions
Using a Cloudflare Firewall to Reduce Server Load
CodingCloudFlarefirewallhostingSiteGround
How Cloudflare's free firewall allowed me to cut CPU usage by over 90%.
Show full content

I host this site (and several others) using SiteGround’s managed WordPress hosting service (disclosure: that’s an affiliate link). They’ve provided great service for years, and I’m happy to stick with them.

A while back, they sent me an email to say that my account was nearing its monthly CPU usage limits. Nothing here is exceptionally high volume on a normal day, so I suspected there was something nefarious afoot. After some searching through the file system, I couldn’t find any evidence of a hacked site or a spam relay, so I started browsing through the reports and statistics in my account admin to try to narrow things down.

What particularly stood out to me: this very domain, xplus3.net, was receiving millions of hits a month, significantly above the few hundreds to thousands of visitors my analytics tell me come to the site in the same span. After drilling in a bit further, almost all of those requests were to /wp-login.php. You might be surprised to learn that I, the only author on this infrequently updated website, do not log in millions of times a month; thrice would push the bounds of credulity. Someone is trying to brute-force their way into my site.

With the problem sufficiently identified, I needed a system to stop all of that traffic to the login URL, while still allowing myself to log in when necessary (stop laughing, I know I should write more). More out of curiosity than any real technical need, I’ve had this site proxied through Cloudflare’s free plan for several years (about as long as I’ve been with SiteGround). Maybe, thought I, there’s a way to set a firewall on Cloudflare that could mitigate this ongoing threat.

Delighted was I to find that the solution was just a few clicks away. I set up a rule to match any traffic to /wp-login.php. Before a visitor makes it to my host, there’s a brief (approximately 5 second) delay while Cloudflare decides if I’m a real visitor. Traffic to the login page stopped immediately, and SiteGround is much happier with my CPU usage.

An example of Cloudflare firewall rules filtering requests to wp-login.php

I haven’t had any issues with the JS Challenge filter. I did try the Captcha option but found it too difficult to prove myself human, so it looks like this filter is my best option for now. The stats are showing me that I should probably address xmlrpc.php next.

http://xplus3.net/?p=622
Instant Braintree Transaction Settlement
CodingAPIsBraintreecredit cardstestingWooCommerce
I’m working on an e-commerce project using WooCommerce and the Braintree payment gateway. One of the areas I’m testing involves issuing partial refunds on orders (e.g., one item out of a larger order is canceled). Part of the credit card transaction process involves submitting an order for settlement, then waiting for that order to settle … Continue reading "Instant Braintree Transaction Settlement"
Show full content

I’m working on an e-commerce project using WooCommerce and the Braintree payment gateway. One of the areas I’m testing involves issuing partial refunds on orders (e.g., one item out of a larger order is canceled). Part of the credit card transaction process involves submitting an order for settlement, then waiting for that order to settle with the credit card processor. You cannot issue a partial refund until the transaction is settled. This process can take anywhere from 30 minutes to 24 hours, an inconvenient bottleneck to development and testing.

Fortunately for us, there is a route to circumvent this delay. The Braintree API provides a method to immediately settle sandbox transactions.

In the context of WooCommerce, we can take advantage of this method while still using the API connection established by the Braintree plugin. This can be run in the context of a WP-CLI shell session, built into your own custom CLI command, or called from an automated testing suite.

// get the ID of the transaction you want to settle
$transaction_id = 'your-id-here';

// get the instance of the WooCommerce payment gateway for braintree
$gateway = wc_braintree()->get_gateway( 'braintree_credit_card' );

// use the WooCommerce gateway to authenticate with Braintree's SDK
$sdk = new \Braintree\Gateway( [ 'accessToken' => $gateway->get_auth_access_token() ] );

// call the testing API to immediately settle your transaction
$transaction = $sdk->testing()->settle( $transaction_id );

Your transaction’s status should now be “settled”, and your order is ready for issuing partial refunds.

http://xplus3.net/?p=616
Extensions
Augustine on the relationship between scripture and science
EtceteraAugustineGenesishumility
We should not rush in headlong and so firmly take our stand on one side that, if further progress in the search of truth justly undermines this position, we too fall with it.
Show full content

From The Literal Meaning of Genesis by Augustine, translated by John Hammond Taylor.

Book One, Chapter 18

In matters that are obscure and far beyond our vision, even in such as we may find treaded in Holy Scripture, different interpretations are sometimes possible without prejudice to the faith we have received. In such a case, we should not rush in headlong and so firmly take our stand on one side that, if further progress in the search of truth justly undermines this position, we too fall with it. That would be to battle not for the teaching of Holy Scripture but for our own, wishing its teaching to conform to ours, whereas we ought to wish ours to conform to that of Sacred Scripture.

Book One, Chapter 19

Usually, even a non-Christian knows something about the earth, the heavens, and the other elements of this world, about the motion and orbit of the starts and even their size and relative positions, about the predictable eclipses of the sun and moon, the cycles of the years and the seasons, about the kinds of animals, shrubs, stones, and so forth, and this knowledge he holds to as being certain from reason and experience. Now, it is a disgraceful and dangerous thing for an infidel to hear a Christian, presumably giving the meaning of Holy Scripture, talking nonsense on these topics; and we should take all means to prevent such an embarrassing situation, in which people show up vast ignorance in a Christian and laugh it to scorn. The shame is not so much than an ignorant individual is derided, but that people outside the household of faith think our sacred writers held such opinions, and, to the great loss of those for whose salvation we toil, the writers of our Scripture are criticized and rejected as unlearned men. If they find a Christian mistaken in a field which they themselves know well and hear him maintaining his foolish opinions about our books, how are they going to believe those books in matters concerning the resurrection of the dead, the hope of eternal life, and the kingdom of heaven, when they think their pages are full of falsehoods on facts which they themselves have learnt from experience and the light of reason?

http://xplus3.net/?p=583
nginx as HTTPS proxy for Elasticsearch
Codingdockerelasticsearchnginxproxy
Since Elasticsearch is exposed via an HTTP API, we can user our nginx server to proxy Elasticsearch requests using the HTTPS protocol.
Show full content

Let’s say you have your local dev environment configured to use SSL. Your dev site is accessible at https://mysite.dev/. Wonderful! Now you need to add Elasticsearch to your project. Let’s add it to docker-compose.yml, something like:

version: "2"
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.2.2
    environment:
      - xpack.security.enabled=false
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
      mem_limit: 1g
    volumes:
      - elasticsearchindex:/usr/share/elasticsearch/data
    ports:
      - "9200"
    network_mode: "bridge"
  # and some other services like PHP, nginx, memcached, mysql
volumes:
  elasticsearchindex:

How do you make requests to Elasticsearch from the browser?

Option 1: Set up a proxy in your app. This probably resembles what you’ll ultimately get in production. You don’t really need any security on Elasticsearch for local dev, but in production it will need some sort of access control so users can’t send arbitrary requests to the server. If you’re not using a third-party service that already handles this for you, this is where you’ll filter out invalid or dangerous requests. I prefer to let more experienced hands manage server security for me, though, and this is a lot of overhead just to set up a local dev server.

Option 2: Expose Elasticsearch directly. Since I don’t need security locally, I could just open up port 9200 on my container and make requests directly to it from the browser at http://localhost:9200/. Notice the protocol there, though. If my local site is at https://mysite.dev/, then the browser will block insecure requests to Elasticsearch.

Option 3: Use nginx as a proxy. I’m already using a reverse proxy in front of my project containers. It terminates the SSL connections and then passes through unencrypted requests to each project’s nginx server. The project’s nginx container doesn’t need to deal with SSL. It listens on port 80 and passes requests to PHP with fastcgi.

server {
	listen 80 default_server;
	server_name mysite.dev;
	# ... more server boilerplate
}

Since Elasticsearch is exposed via an HTTP API, we can create another server block to proxy Elasticsearch requests. First, make sure the nginx container can talk to the Elasticsearch container. In docker-compose.yml:

  nginx:
    image: nginx:stable-alpine
    environment:
      - VIRTUAL_HOST=mysite.dev,*.mysite.dev
    volumes:
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
      - ./nginx/elasticsearch-proxy.conf:/etc/nginx/conf.d/elasticsearch-proxy.conf:ro
      - ./nginx/php.conf:/etc/nginx/php.conf:ro
    links:
      - php
      - elasticsearch
    ports:
      - "80"
    network_mode: "bridge"

And then create elasticsearch-proxy.conf to handle the requests:

upstream es {
	server elasticsearch:9200;
	keepalive 15;
}

server {
	listen 80;
	server_name search.mysite.dev;

	location / {
		proxy_pass http://es;
		proxy_http_version 1.1;
		proxy_set_header Connection "Keep-Alive";
		proxy_set_header Proxy-Connection "Keep-Alive";
	}
}

Now we can make requests to Elasticsearch from the browser at https://search.mysite.dev/. The nginx proxy will handle the SSL termination, and communicate with Elasticsearch using its standard HTTP API.

http://xplus3.net/?p=578