quantblog

Digital Twins are great, but they have some real pain points :

• cost of scanning
• cost of modelling
• massive data

I believe Machine Learning will be used to solve those practical issues.

New ML algorithms will automatically process photos and lidar scan data into 3D models and tag that data – radically reducing the cost to create Digital Twins and increase the value and usefulness they provide.

Digital Twin : Lite 3D models + extra tags

At its heart a Digital Twin is an accurate 3D model, with extra information to make it useful as a proxy for the real world. There are two basic methods to make a 3D model of a building, public space or industrial plant :

• LIDAR laser scan creating billions of dots in 3D space – a pointcloud
• photogrammetry – taking hundreds of photos from a drone or camera and generating a fine 3D textured mesh

There are some bottlenecks with those current methods :

• LIDAR scanners are expensive, require skill and software to operate [ 30k device, 5k/yr software ]
• LIDAR scanners produce a firehose of data – pointcloiuds are often 20GB to 200GB in size
• turning a LIDAR scan into a 3D CAD model is a manual labor-intensive process
• Photogrammetry usually requires a large number of photos for good results : 400+ photos
• Photogrammetry generates a fine mesh which has a relatively large data footprint

To actually use that data, we need lite 3D CAD models that contain the essentials in an efficient format. For example, a flat wall is a simple quad plane, a pipe represented by a cylinder – not a million points or hundreds of triangles.

We might want a color or texture or tags that describe the wall or pipe, such as its thickness, material, whether it carries cold water steam or oil, a part inventory number, geolocation etc. The BIM .ifc classes are a formal standard for adding some of this information, and simple tag keywords can also go a long way to making the data more usable / searchable.

Machine Learning to the rescue

I believe we are on the brink of having Machine Learning algorithms that can solve those problems and make Digital Twins more cost effective and widespread – but it does require investment and a dedicated engineering effort to apply current techniques to build these new solutions.

ML solutions can be built to automate the creation of these lite 3D models :

• auto generate 3D CAD models from pointcloud lidar scans
• auto generate 3D CAD models from fine mesh photogrammetry ‘scans’
• auto generate 3D CAD models from 360 panorama photo ‘scans’, for cases where “cm accuracy” is adequate
• auto tag the 3D CAD model geometry, so the data is easy to query by humans, programs and LLM or Generative AI

Given the rapid advancements in ML, I think we will see these solutions emerge in the next year or so. It seems inevitable and when it happens it will lower the cost of Digital Twins, generate a lot of value and unlock a wide range of uses and applications.

Bringing down the cost will make 3D Digital Twin models of the real world become ubiquitous, it will be easy to share lite 3D models over the internet. Imagine :

• if streetview became a full 3D experience, where you can walk into and around any public building
• 3D and 2D detailed floor maps of every shopping center, campus, transit station, airport/mrt
• 3D / 2D maps of current state of every construction site
• remote shared view : design walkthroughs of buildings, safety reviews and work plan for construction sites
• VR/AR/3D web shopping, virtual meetings, design reviews, realestate sales and gallery showroom tours
• fully capture every pipe, rail, I-beam, cable, conduit of industrial plants, warehouses, production lines and engine rooms
• robots trained on lifelike simulations of the 3D environment
• Architects, renovators, tradespeople and designers having accurate CAD models quickly and at low cost

We can build this

These new Machine Learning algorithms can be developed quite cheaply given the value they unlock.

The above ML components could potentially be developed by 4 small teams of say 3 engineers on each project for 6 months. A ballpark cost of 4 x 3 x 0.5 x 100k = 600k USD to get to the MVP Minimim-Viable-Product stage. Afther that, they are likely to attract funded trials by mid-size companies for their Digital Twin projects, driving the commercialization of the technology.

We know the engineering is tractable, here are some of my own experiments that prove out the concepts :

• auto detect geometry in pointclouds : walls/edges & pipes
• model directly over 360 panoramas : pipe centerlines & textured walls

A quick google tells me that the worldwide market for Digital Twin tech is around 20Bn USD growing at 40% per year .. with 5Bn per year being spent on manual scan-to-CAD. Given the economic value, my question is – why arent we developing this technology full-steam ahead ?

It seems such an obvious and practical use of the new ML techniques and GPU compute – building this missing technology will unlock a vast set of useful applications, bring 3D to the web and reduce the cost of Digital Twins. We need a rich visionary to bet on the future and write a term sheet to get it done.

Overview

There is an accelerating arms race brewing over low-latency Machine Learning in trading firms –
those who use ML to detect market microstructure and behavior in realtime and quickly simulate a
trade outcome then execute it will have a competitive advantage.

ML trading is complicated by the need for sequential rounds of GPU Neural Network inference and
CPU simulation / evaluation immediately before trade execution.

This has implications for hardware and software architecture :
Current and planned generations of datacenter hardware, such as VeraRubin, have CPU and GPU
on distinct packages. These systems have impressive compute and bandwidth but relatively high
latency when compared to fast trade execution.

In contrast, we now have examples of single package CPU+GPU hybrids, such as AMD Strix Halo.
The many short wire routes between CPU, RAM cache and GPU, result in minimal latency
between the two.

Two phases of compute : GPU inference and CPU simulation

The game of chess is a good analogy to explain these two phases – to play chess well you need to have lots of learned experience, but also simulate ahead before you take the next move.

You are likely to encounter states of the board you haven’t seen before. Learned experience can
suggest good moves and rule out bad ones, but then you need to simulate and play ahead a few
moves to check your intuition, rule out weak moves, find traps, and decide on the best choice
amongst the top candidates. Speed chess is even closer to the problem of trading – you have
limited time to simulate ahead.

When computing a best move or trade, we need both CPU and GPU :
Learning from experience and inference run in a Neural Network on the GPU
Simulation/Evaluation is domain model code that runs on the CPU

Why do we need inference? Because the domain is too large and complex to compute
exhaustively, there wouldnt be time to visit enough states.

Why do we need simulation / evaluation ? Because the domain / state space is so large that you
havent seen the current state ever before – intuition is better than no plan, but it needs to be
checked against the current state.

Why do we need to do these two phases sequentially ? Because first you need to infer or guess
some good moves or trades based on the current state of play, and only then can you simulate and
evaluate those actions to see if they are productive.

Why cant we run everything on the GPU ? Technically most GPUs are Turing complete, but they
are not well optimized for branching, whereas modern CPUs do a lot of branch prediction /
speculative execution. Then there is the practical issue that most model code is written for CPUs.

Many complex hard domains contain inverse problems, where it is hard to guess or compute the
underlying hidden state, but the observable state is relatively easy to score or compute from the
hidden underlying state, similar to one-way functions in cryptography, these usually run on CPU.The ML Compute tree

The basic compute block looks like this :

The inference step takes in the current state and suggests several actions, each with a probability
of winning or expected score value. The simulation step evaluates the model and provides a
score estimate.

We also want to look ahead and simulate further than one step into the future, so we explore a tree
of inference and simulation/evaluation branching out from the current state. We prune the tree
when we reach places that have a low eval score.

The issue isnt the amount of total compute – we can use multiple CPU and GPU cores to do many
compute blocks in parallel. The problem is the total latency, which is dictated by the depth of the
tree.

At N levels deep we have latency of ΔT = N * (Δt GPU + Δt RAM + Δt CPU + Δt RAM).

Hardware CPU-GPU example

Here is an unofficial annotated die shot of AMD Strix Halo architecture with main CPU, GPU and
cache areas highlited, with 16 CPU cores [32 threads] and 20×2 GPU cores on the larger main tile.

The two CPU tiles at the top are each connected to the larger GPU die via physical wires in a fan-
out / fan-in architecture in the yellow box regions. Earlier versions used a serdes interconnect, but
many physical wires resulted in lower power and lower latency.

Related Concerns

For some domains we might be able to use momoization to reuse results from the previous round,
but with large or continuous state spaces, we might not be revisiting any of the prior computed
states.

In classic model-centric Reinforcement Learning, the results of forward simulation are used to
correct the inference weights. The NN starts with random guesses and learns to predict the model
behavior via feedback. The model can be simple but the inverse problem hard, in which case a
simple implementation of the model running on the CPU has trained the NN to solve its inverse.

It may be the case that the NN is so good at predicting actions that we no longer need to simulate
forward. Even in that case it may be useful to have model simulation as a way of mitigating risk
against the NN giving bad predictions in some edge cases. Another possibility is the drift of the
domain into a new regime the NN is no longer well adapted to.

If the ideal hardware has CPU, Cache and GPU in close proximity for low latency, we need
software to utlize that low latency – for example a lockless userspace ring with atomic
synchronization barrier and fitting data into cache lines so it can be moved quickly between CPU,
cache and GPU.

Other engineering considerations include:
   How often to train / retrain the NN and update the model ?
   What is the latency budget and how many levels of iteration are enough to give good results ?

There is a trend for LLMs to do more inference time or edge-compute, be it processing more
context via RAG, or performing more logic or formal rule validation, or being augmented with
elements of RL. If this trend of more edge compute continues it might be the case that general
purpose AI will run best on balanced CPU GPU hardware, with low latency path between.

Moores law may have slowed, but the new compute needed for soft realtime problems luckily
matches the many core GPU+CPU hardware we now have. Modern ML / RL apps in engineering /
logistics / finance may be the killer apps for modern balanced processors with high GPU, NPU and
CPU core counts located in close proximity.

Future Work

It would be ideal to compare performance of some simple problem domains across two current
generation hardware platforms to establish actual latency numbers. This would help estimate the
impact of colocating the CPU and GPU in the same package, on modern ML workloads, ie :

Test each of these sample Problem Domains :
   Trading order book : predict hidden orders
   2D vectorization : reconstruct image from simple line,circle,box,gradient primitives
   Simplified ARC style problems : finding combinations of simple operations to match examples
on each of these commodity hardware platforms :
   AMD Strix Halo 395 with CPU and GPU in same package
   NVIDIA Blackwell or GraceHopper with distinct CPU and GPU

References

https://www.techpowerup.com/332745/amd-ryzen-ai-max-strix-halo-die-exposed-and-annotated
https://en.wikipedia.org/wiki/Monte_Carlo_tree_search

author : gord anderson : justgord at gmail dot com

There is chum in the water .. LLM mania is sucking up all the oxygen, all the deal flow and all the GPU burn .. in the fomo full-metal jacket panic attack that surrounds the question : who will get to general AI first ?

We’ve all been pitched three LLM-wrapper startups this week.. but if LLMs work a little better, they will absorb all the wrapper apps and APIs.

Yet, now is the best of times to be an investor .. LLMs are not the full story of the Neural Network AI revolution – and the noise around them is drowning out and sucking cash away from other very worthy hyper growth startups. As a deep tech enthusiast I look around and see signs of a quiet revolution at the edges – the growth in every area of technology and engineering is about be multiplied by 50x over the coming decade as Reinforcement Learning techniques get applied to every engineering and manufacturing domain – from chip layout to logistics to medical treatment plans to building-site processes.

Why are LLMs not everything .. smartest wins, right ? No matter how smart you are Nature can outfox you – the math of the butterfly effect in complex systems still holds. For many engineering processes that relate to the real world – material science, protein folding, biological markers of ageing, packing containers, crop yields, turning photos of a building into a 3D model – what matters is how many millions of combinations you can try to find a great solution, and how you can encode intuition of that. In some cases there provably is no best solution attainable .. and the quantum computers that can solve these by collapsing a wave-function of infinite possibilities, wont be here for a while. Think of RL as the slow approximation to a quantum computer, which you can invest in now .. if it is applied to solving a real problem that is hard for humans and has actual dollar value, backed by a capable team with a clear vision.

Reinforcement Learning .. or as I call it “simulation learning”, is essentially the mix of a Neural Network to learn intuition and a Monte Carlo simulation to explore many of the possibilities, to walk around the search space. If you want a computer to reach super human level of Go or Chess, or even Tetris, play, then due to the vastness of the search space, you need to play many moves ahead to weed out bad options and test the strength of a good next move. You cant explore all the options ahead of your next move, so you have some randomized strategy which is a tradeoff between depth and breadth.. you play ahead in your mind and see what happens. You then encode that intuition and compress it into the Neural Network .. it learns to not make dumb moves, it learns good strategies, such as the relative value of pieces and areas of the board. It compresses the experience of all that exploration into internal rules and behavior… its not intelligence, its experience, the result of exploration.

One technical point – a lot of that simulation legwork runs well well on a CPU with lots of threads and fast RAM. Simulation learning is well suited to the kinds of balanced architectures we see in the current crop of chips. Interesting to see Lunar Lake has this balanced set of features : multicore CPU, NPU, GPU and RAM in close proximity. Its possible the “killer app” for this new generation of “AI / copilot” capable PCs is not an LLM personal agent, but an app in a medicine / logistics / 3D design / construction domain.

TL/DR

So.. If your looking for investment opportunities which are not overvalued by LLM hype, and which have great scalability and growth potential .. I would look to those startups which have a combination of great engineering and RL tech talent applied to a real world problem domain … that is where the undervalued hyper growth is at the moment.

Some other fun discussions around this topic :

Some other discussions around this :

• Anastasia In Tech re : Googles AlphaChip layout AI
• Sabine Hossenfelder re : LLMs are meh
• HN discussion on GPU-rental bubble
• SequiaCap on AIs 600Bn Question
• Asionometry and DylanPatel animated freewheeling discussion

Im not saying LLMs aren’t astounding and will lead to massive productivity gains .. they are and will. I am saying keep an eye out for under-the-radar real-problem RL startups, because they have great impact and value creation.

We are in the internet boom of AI .. the era of pets.com… noise-to-signal ratio is high.

Team tiyuti – gord and paul – have been busy building a new web shop technology, using the same deep indexing tech we used to search 40k Melbourne restaurant dishes ..

demo shop here : “Classic Watch Men” : https://demo.tiyuti.com

Built with hand-crafted javascript : node.js and postgresql under the hood.

[ Looking for people who have a few thousand products to sell, to showcase our tech .. emailing support@tiyuti.com will reach the founders :p ]

#ecommerce not #shopify not #woocommerce #webstore #webshop #indexing #nodejs

From the previous article, a few people mention 10 writes per second is pretty damn slow – and they have a point.

So I tried pushing things to the edge of where performance starts to degrade …

I ran a whole lot of writer processes and some reader processes .. and I see a steady 120/sec WRITES while still having good response to reads / low latency.

Beyond that, if I push it up to 160 to 200 WR/sec I see read performance degrade ..ie. instead of latency being <80ms .. they blow out to 250 .. 500ms .. then go up dramatically as the system has to queue reads and writes, buffers them and tries to catch up.

So I guess the best conclusion I can draw is that out of the box performance node.js+postgresql under simulated real-world usage, gives you around 120 writes per second while not degrading performance.

Thats not too bad .. 100 writes/sec would be a lot of users hitting your average web site .. but admittedly there are a lot of optimizations one could make.. this was more about how far you can get without doing any clever performance tweaks [ redis, message queues, server sharding yadda yadda ] ..

Well, the MON idea was really about getting visibility into host latency .. once you do that you then have actual data, that may or may not match your intuition.

tldr : I pushed writes to 120 wr/sec before seeing degraded performance

At tiyuti.com, we’re testing out general performance and usability of PhoTree : share your list ! .. so imported some realworld data to try things out :

• Something to read ? search 10k books here : Big Famous Book List

• Something for Lunch Dinner Date ? search 30k dishes : Melbourne Food Finder

Performance seems pretty good – both pages scored around 95 when tested on pagespeed.web.dev

Content feels like its loaded after second or two .. not too shabby for a fully javascript SPA [ single page app ] which gets all its data as json then generates the page elements… and there are plenty of places to reduce latency. One thing we are glad we did is gzip compress, which helps reduce a large json data payload footprint.

Been busy building PHO aka “PhoTree : share your list” a kind of photo list link tree. The idea is simple, you make a list of your cool stuff and share it, as easily as possible – no login, no email, no signup process – just make a new list and start adding items with pics to it.. within seconds.

We wanted all text to be automatically indexed so you can search anything – ie. you can search for a book by author, title, isbn topic / whatever. We imported a few thousand books as a test – check out the “Big Famous Book List” if you’re looking for something to read

We [ me and my son Paul, on holiday from uni .. aka ‘Team Tiyuti’ ] started with the web json api first and built PHO over the past week and a half of hacking javascript and css. Instead of the usual register/email/login/password guff, we went for a secret key needed to edit a given page – so we don’t harvest peoples information and you can make a new page instantly.

We built it using node.js and postgresql .. and I was asked a few weeks back by a colleague “is node.js fast? ” .. my impression is its pretty damn fast, for a scripting language. Later I thought well, it would be nice to actually listen in on all the web and json api endpoints, and get latency stats … ie. evidence rather than hearsay / intuition.

So.. I hacked up MON – a latency monitor for node apps. MON wraps your node.js http handler, looks at the request coming in, and the response going out and measures the time delay ‘dt’ or ‘latency’. It keeps track of latency and counts for each api endpoint url that comes through and serves up a summary page.

I couldn’t resist adding the blinken lights that flash green when a request comes in : )

To get an idea of the performance we can expect from a web app built with node.js and postgresql database on a common or garden VPS server with SSD disk .. we need to put the system under some simulated user load. So I wrote reader and writer test scripts to make pages and items and fill them with random words, and read them back in any order etc .. the above tests shows a fairly heavy user load of ~10 writes per second and 20 reads per second .. and even then the median latency is well under 50ms.

I did notice a few outliers at around half a second .. which may be due to warming up the DB cache, or possibly garbage collection .. which I will look at more in future. The human eye takes around 200ms to blink, and if the roundtrip + page rendering is in the order of a second, most users will experience that as very responsive.

My gut feel is that node.js itself is not doing a lot of work, basically acting as a router, sending requests for files to linux filesystem and sql requests to the database .. so what we are seeing here is basically excellent performance by postgresql, linux and modern SSD backed hosting environments.

If you enjoyed this read, you might also want to have a look at another very terse article on a similar topic : “web search 30 million rows sub-second, cheaply”

enjoy : gord at tiyuti dot com

ps. you can hire us to build your new new – but fast – thing :p

Here are my thoughts on sitting the British “A-Levels” in Australia as a possible path to University Entrance for Home Educated students.

Disclaimer :  This is a guide only – our current plan which we have not yet verified/ completed – so please check for yourself before making any decisions.

Questions :

• What are A-Levels ?
• Why do them ?
• How to do them ?
• Costs / Logistics ?
• Does it really work ?

In a nutshell, the A-levels are the UK / British approximate equivalent of the Australian VCE/HSC/ATAR, or the American CollegeBoard/AP exams, or the IB International Baccalaureate.  That is to say, they are the highest secondary school qualification, and are used to gain entrance to University and satisfy tertiary prerequisites.

A-levels are sat in many international schools worldwide, and are quite well known by universities in many countries, so the qualification is reasonably well regarded and can usually be compared/converted to the local equivalent.  The exams are standalone and not based on coursework, they are marked by an independent body and have few formal prerequisites or formal age requirements.   There are plenty of course materials, books and past exam questions available, and A levels cover a wide range of subjects in depth.

In Australia, the BritishCouncil in Sydney run the exams in May/Jun and Oct/Nov, and some in Jan.  There are various exam “boards” which set the exams and mark them, and BritishCouncil offer the option of EdExcel or CIE ‘flavor’ exams.

In our case, DS15 – code for ‘my 15 year old Darling Son’ – has worked a couple of years ahead in Math, using materials from KhanAcademy.org, AoPS.com and amt.edu.au.    Doing some fairly rigorous academic tests in Math next year seems a good fit, in preparation for University the following year.   If we could have done VCE methods, ‘spesh’ and physics next year that would have worked out fine – but we could not find a school that would allow us this flexibility.

In the UK it is normal to sit 3 A-levels, so our plan for 2019 is roughly :

• “Maths” in May/Jun
• “Further Maths” in Oct/Nov
• “Physics” in Oct/Nov

I think this gives us a reasonable chance of getting into a good science/math program at an Australian university, but they may insist he also sit an entrance STAT exam to show evidence of English / Reasoning / Writing skills at university level.

The A level exams are not super cheap – each test is around $300 AUD, with usually 2 or 3 tests per subject on different days, so around $2400 in our case.  There is also the issue of travel and accommodation to sit the tests in Sydney and lesser costs for books and materials.  There are some online distance ed providers for A levels, but in our case we prefer to self study from the books.

If things go well, we will report back on our success and this pathway might prove useful for other home educated students.  If things don’t go well, we can review what went wrong, and DS will still have time to find other pathways to get into Uni.   I admit this is a pretty “geeky” approach with so much emphasis on Maths, but it suits us well and DS does have wider interests – he plays the violin, reads voraciously, writes, does computer art and programming, loves cooking etc.   Even with good marks in A-levels a 16yo student will most likely need the University Dean of Schools approval, so we shall see what they say about our best-of-british quals !

Some links :

• British Council Australia IGCSE and A-Levels
• Pearson Edexcel A-Levels
• CambridgeInternational / CIE A-Levels

Initial beta release of my instant web wallet for Bitcoin Cash : moosh.io

I talked to a few people about getting started with BCH, but they had issues setting up a wallet to accept payments.

The first time you visit moosh.io, it creates a random wallet address for BCH, so you can receive funds and send them on.

The other problem is how to accept payments as a small business – a cafe, restaurant or food stall.  Basically you need a simple way to know which order/product/sale matches up to which payment – who has paid for what.  I achieve this by having a temporary BCH address to accept the payment for a given order, and then forward it on to the main vendors wallet as soon as payment is detected.

All the keys are kept on the client side, in localStorage on the local device, and transactions signed within the browser.

Ill be extending this to look nice and work well on mobile devices – now that money is purely digital, all those heavy point-of-sale hardware can be replaced by a nice web app running on your phone.

I was discussing with other parents online, about the merits of various school Math multiplication approaches.

In the “Japanese Line method” you draw lines for each digit, and then count the intersections and add them up.

In the “Box-Method” you draw areas for each sub-product, and in “Longhand Multiplication” you just use the square grid to keep track in a table, without use of a diagram.

My main issue with the Line Method is that for problems like 97*98, you’ll be drawing a lot of lines .. its a lot of work even for 64×34 :

I also like the idea that Box Method starts with a physical / visual intuition – you can start children counting rows and columns, then progress to counting areas such as “How many 1ft square tiles do I need to tile the 3’x7′ kitchen floor ?”

Then you can discuss the distributive law a(b+c) and (a+b)(c+d) and explain the method behind longhand multiplication .. and then go on and show some handy tricks, such as using negative numbers to make less work, vis :

The above example, which does the same problem in 2 different ways, shows the kind of approach Jo Boaler recommends, called “Number Sense” – where students are encouraged to find their own way of doing a math problem.  The idea is that there are many valid ways to do it, and struggling to find your own way is a good thing [ even if you fail ], and students tend to engage more, and certainly recall more when they have actively discussed with other students.

The same Box Method diagram can lead onward naturally to algebra and quadratics.. and even the beginnings of calculus