Welcome to ned Productions (non-commercial personal website, for commercial company see ned Productions Limited). Please choose an item you are interested in on the left hand side, or continue down for Niall’s virtual diary.
Niall’s virtual diary:
Started all the way back in 1998 when there was no word ‘blog’ yet, hence ‘virtual diary’.
Original content has undergone multiple conversions Microsoft FrontPage => Microsoft Expression Web, legacy HTML tag soup => XHTML, XHTML => Markdown, and with a ‘various codepages’ => UTF-8 conversion for good measure. Some content, especially the older stuff, may not have entirely survived intact, especially in terms of broken links or images.
- A biography of me is here if you want to get a quick overview of who I am
- An archive of prior virtual diary entries are available here
- For a deep, meaningful moment, watch this dialogue (needs a video player), or for something which plays with your perception, check out this picture. Try moving your eyes around - are those circles rotating???
Latest entries: 
Word count: 3807. Estimated reading time: 18 minutes.
- Summary:
- A house build is described, alongside a need for temporary accommodation due to eviction. Mobile homes are analysed, with brands like Willerby, ABI, and Swift compared. Insulation levels and residential specifications are evaluated. A plan for a hardstanding area is detailed. Finally, the depreciation of mobile homes and the progress of an AI project are mentioned.
Tuesday 15 September 2026: 11:43.
- Summary:
- A house build is described, alongside a need for temporary accommodation due to eviction. Mobile homes are analysed, with brands like Willerby, ABI, and Swift compared. Insulation levels and residential specifications are evaluated. A plan for a hardstanding area is detailed. Finally, the depreciation of mobile homes and the progress of an AI project are mentioned.
- Foundations should start to be installed end of this month.
- Timber frames should be installed early February.
- We received notice of eviction from our rented house today, and we have to out by March.
Obviously the house won’t be ready to move into by March, so we need somewhere to live for a few months to bridge the gap. As landlords don’t look kindly on renting out to those without income, and given the current Irish rental market, I would expect our chances of finding a new rental to be approximately nil. One could rent a holiday home from the likes of AirBnB, but that would get real expensive over the summer, and money isn’t exactly growing on trees right now. So we probably have no choice: we’ll need to live in a mobile home (aka ‘static caravan’) for a while.
Ugh, mobile homes
I’ll start by saying I absolutely hate camping, and a close second behind my hatred of camping is caravans and then mobile homes. Back in July 2021 when covid was raging and there were no vaccines, we rented a campervan to get to and from Megan’s brother’s wedding in England without risking catching covid, and thinking about those three weeks still gives me the shudders – I remember myself and Megan definitely had PTSD for days after our return, though the kids loved it. Those awful contraptions are either too hot or too cold, they are always damp so I can’t sleep, and they always smell bad. They’re uncomfortable, cramped, and exactly the last thing I’d want to endure during a holiday. So, given my hatred of all things camping related, I have never had the slightest interest in mobile home nor caravans other than a vague awareness that a ‘Pemberton’ was a slightly less awful brand for twice the price of a normal mobile home.
Obviously now I need learn about them in quite some detail so I have some idea what to look for and what tradeoffs to make in terms of age to features to price. But firstly, here is where I intend to place the mobile home on the site:
We were going to be extending the hardstanding in any case where the white shaded patches are to permit low loader trucks turning access next to the house – we were prevented because right now there is a giant soil pile there which still hasn’t been taken away despite me ringing them weekly. That same extended hardstanding happens to be where the patio will go in any case, so we’re just doing it early. We have a 3.6m wide gap between the house corner and the shipping container, and as lorries can be a maximum of 3.0m wide they can get past with a bit of careful driving.
My plan is to dig a shallow 250 mm deep hole where the pink shaded patch is – this is 15.5m wide and 5.5m tall, which should accommodate a mobile home sized 11.8m long and 3.76m wide with a 1.5m wide hardstanding at the front. That is 21.3 m3 of volume, so about 42 tonnes of crushed rock, which is two eighteen tonne truck loads probably of four inch down laid 100 mm thick each layer, with six tonnes of two inch down or two inch clean finishing layer making up the top 37 mm. By being right next to the rear walls, hopefully the prevailing south west wind will mostly flow over the mobile home, reducing forced air changes per hour. There should be a 6m gap between the mobile home and the foundations at their closest point, easily enough for cranes etc to pass.
We have existing electricity right next to that location, and foulwater and mains water would only need a spur about 18m long to be added. That’s well within my own personal capabilities, so I’d just rent a digger and do it myself:
You get about 40 m2 of floor space in a three bedroom 12 ft wide mobile home. It’s not a lot, however after February 2027 will we have at our disposal 250 m2 of indoor space in the house being constructed, plus the office portable cabin remains available. Our current rented house has about 55 m2 of floor space, so the mobile home will be equivalent to losing our front downstairs room which I currently use as my work office (and is mostly filled with storage in fact). And if we get a mobile home with an equivalent insulation level to this rented house, the adjustment may not be as awful as it first seems – we’ve effectively already been living in a mobile home until now.
What we will need is temporary storage as the shipping container at the site is full. Thankfully, North Cork has some of the cheapest self storage in the country: about β¬6 inc VAT per month per m3 of shipping container storage. Once the house is up and secure, we can move everything out of self storage into the house.
Detangling mobile homes
I’ve learned far more about mobile homes than I ever wanted to in the last few days, so first fun fact about them: a shockingly large percentage of the total British/Irish stock are made in and around Hull, Yorkshire, where I once used to live. I had no idea if I’m honest: there is a genuine cluster of manufacturing around the East Yorkshire region, and the three most popular brands by volume are all made there, along with several of the small upstarts which are startups created by former employees of the big manufacturers. Over in Europe mobile homes and caravans are quite different to the British ones, over there they tend to build things to last with an accompanying high initial purchase price, whereas in Britain/Ireland they are built to a cheap standard and build quality is more uneven – however they are much cheaper as a result.
Speaking of which, here are the top brands by market share, and my single line summary of each:
- 33% Willerby – ‘nobody ever got fired for buying a Willerby’.
- 13% ABI – slightly better bang for the buck over a Willerby.
- 10% Swift – slightly better bang for the buck over an ABI and slightly more again over a Willerby.
- 8% Carnaby – budget choice for those who can’t afford a Willerby.
- 36% Others (in order: BK Bluebird, Cosalt, Pemberton, Delta, Victory)
At least in the British/Irish market, everything is defined by Willerby, and everybody else measures themselves against Willerby. Willerby, being large and conservative and the biggest, moves slowly and conservatively, but they set the pace and pricing for the rest of the market. They probably also have one of the best after sales support, from what I can tell, followed by Swift where there are more reports of build quality issues but after sales seems good at getting them fixed, then quite a large gap to ABI where after sales appears to be poor, with Carnaby at the bottom with essentially no after sales support at all and you’re on your own to fix any build quality issues – on the other hand, you have saved β¬8-12k and that’s why your mobile home is so cheap.
It turns out that almost nobody tells you specifications or build details, and the brochures are almost entirely pictures of the insides with soothing words about how great the product looks. Willerby is easily the worst on this: they won’t even tell you the exterior dimensions apart from a general ‘this is the largest footprint’, or what insulation is fitted other than ‘twice as much as the previous model’ which is nearly useless. Swift is probably the market leader on detail, you get exact dimensions inside and outside, and also exactly how much insulation is fitted to walls, the floor and roof. They also – uniquely – provide free to download every owner’s manual by year going back to 2001, and from those I was able to build a timeline of what specs each year of Swift mobile home has. Which is exactly what they all should do, to be honest.
Why they don’t is because mobile home sales are an unusual business. Most sales go to caravan parks who then resell them on to clients or they rent them out. Clients who buy them face steep annual ground rental fees of about β¬3k per year excluding bills and insurance, plus you are generally required to remove your mobile home from the park once it is twenty, fifteen or even ten years old, depending on the ‘fanciness’ of each mobile home park. Of course there is a fee for that, and there is a further fee for bringing in your own mobile home. To be honest, I personally would run a mile from any of those arrangements – even if you live in your mobile home three months of the year, the sums just really don’t add up with such steep depreciation and cost overheads. You’d be far better off buying an apartment, and at least that would hold its value and might even appreciate. In any case, because the real customer is caravan parks, and they care about profits rather than specifications, there is little incentive for manufacturers to publish or make available detail about what you are buying.
All that just said, I have reverse engineered some specifications and detail about the big three mobile home brands. I left out Carnaby because it’s rare they are fitted with the optional insulation upgrade, however they’re also transparent: the standard 2025 model has 25 mm wall insulation, and the ‘ultra warm’ upgrade increases that to 50 mm. Until recently, they fit no insulation at all as standard, they were the last to start fitting some out of the major brands. The second last is actually none other than Willerby …
Willerby mobile homes
The cheapest three bed Willerby, the Rio, costs β¬59k new in Ireland and probably has 30 mm of wall insulation (I can find absolutely no data online on what floor or roof insulation is fitted, other than there is some on both). From September 2022 onwards Willerby have been banging on about their ‘GreEN standard’ which is supposedly way higher insulation. In fact, it’s almost entirely marketing fluff:
- 2020-2021: 3.982 wide incl gutters (0 mm insulation per wall?)
- 2022: 4.020 wide incl gutters (9 mm insulation per wall?)
- 2023-2024: 4.042 wide incl gutters (20 mm insulation per wall?)
- 2025-2026: 4.062 wide incl gutters (30 mm insulation per wall?)
I’m purely guessing here by stated width including gutters which is the only datapoint I have, and also their marketing claims about ‘double thickness wall insulation’ and ‘28% energy efficiency improvement’ where I assume that they don’t lie. So while yes while 2022 added insulation for the first time, and yes from 2023 onwards they had double the insulation AND a vapour control membrane, from 2025 onwards they also silently increased the insulation by 50%.
Conclusion: Don’t buy a used Willerby before 2023 (three years old), not worth it. A 2023 Willerby costs about β¬50k right now, and it has less than half the insulation of an ABI after 2020. Willerby before 2023 are considerably cheaper.
ABI mobile homes
The cheapest three bed ABI, the Beverley, costs β¬65k new in Ireland. The extra 10% price bump does get you some nice things however: galvanised chassis (an optional extra for the Willerby), 50 mm wall insulation, 100 mm roof insulation, and an unknown amount of floor insulation. Their history over time:
- 2015: 3.68 wide (3.95 incl gutters). 100mm roof insulation, no floor insulation, unsure wall insulation.
- 2020: 3.68 wide (3.95 incl gutters). Brochure claims 50 mm on all modelβs walls except the Coworth and Rydale, and 100 mm roof insulation on all models.
- 2022-2025: High end is 3.74 wide (4.01 incl gutters); low end is 3.68 wide (3.95 incl gutters). Brochure claims 50 mm on all modelβs walls except the Coworth, and 100 mm roof insulation on all models.
The reason I can’t give you detail between the years is I couldn’t find brochures nor catalogues except for the years 2015, 2020, and 2022.
Conclusion: Don’t buy a used ABI before 2020 (six years old), not worth it. A 2020 ABI with Winter Pack costs about β¬40k right now.
(A ‘Winter Pack’ simply means it has double glazed windows and central heating which usually takes it into EN1647 Grade Three – see below)
Swift mobile homes
Swift like ABI tend to rotate in new models in over time, and carry on selling the old models for a period as the lower end models. Their current new models are the Burgundy, Morlaix and Margaux, and all three have galvanised chassis’, 60 mm of wall insulation, 100 mm of roof insulation, and 80 mm of floor insulation. As I mentioned above, Swift publicise exact numbers for all their models AND provide all the documentation going back decades, so I can make this high fidelity history over time:
- 2005-2015: 3.66 wide (10 mm insulation per wall). Same as a 1950s Irish house, about BER G rating.
- 2016-2024: 3.70 wide (30 mm insulation per wall). Same as a 1980s Irish house, about BER D rating.
- 2025-2026: 3.76 wide (60 mm insulation per wall, but only Burgundy, Morlaix and Margaux), 80 mm in floor, 100 mm in roof. Same as a 1995-2005 Irish house which was BS3632 residential spec 1995-2005, about BER C rating.
Current prices for a new latest generation three bed Swift:
- β¬61k: Burgundy
- β¬66k: Morlaix
- β¬68k: Margaux. This one is unusual in that I don’t find its insides hideous like any mobile home I’ve been in. It actually kinda looks pretty good.
Conclusion: Don’t buy a used Swift before 2016 (ten years old), not worth it. A 2016 Swift with Winter Pack costs about β¬35k right now. Slightly older Swifts are much cheaper.
All brands – the BS3632 residential spec
Leisure mobile homes meet the EN1647 standard. This has grades one, two and three:
- Total building fabric must have a u-value <= 1.7 W/m2K.
- Total building fabric must have a u-value <= 1.7 W/m2K and outer pipes lagged.
- Total building fabric must have a u-value <= 1.2 W/m2K and outer pipes lagged.
My future house has a total building fabric u-value of 0.18 W/m2K I believe – the maximum permitted by Passive House is 0.20 W/m2K. So obviously a leisure mobile home will be rather less energy efficient π.
Everybody says if you want to live in a mobile home over the winter, you need to get a residential spec edition – this raises the insulation and thermal bridging up to the same as a minimum build standard house in Britain, which is the BS3632 standard. Britain used to raise its minimum build standards in lockstep with Ireland until Brexit in 2020, after which they diverged:
- 1989-1994: walls u-value <= 1.0 W/m2K, floor u-value <= 1.0 W/m2K, roof u-value <= 0.60 W/m2K.
- 1995-2004: walls u-value <= 0.60 W/m2K, floor u-value <= 0.60 W/m2K, roof u-value <= 0.35 W/m2K. About Irish BER C.
- 2005-2014: walls u-value <= 0.50 W/m2K, floor u-value <= 0.50 W/m2K, roof u-value <= 0.30 W/m2K. Walls same as pumped cavity block wall.
- 2015-2022: walls u-value <= 0.35 W/m2K, floor u-value <= 0.35 W/m2K, roof u-value <= 0.20 W/m2K. About Irish BER B.
- 2023-2026: walls u-value <= 0.30 W/m2K, floor u-value <= 0.26 W/m2K, roof u-value <= 0.20 W/m2K.
Now, you can kinda map the mobile home specs above to BS3632 as follows:
- 30 mm of XPS would have a u-value of 1.1 W/m2K.
- 60 mm of XPS would have a u-value of 0.55 W/m2K.
- 80 mm of fiberglass wool would have a u-value of 0.55 W/m2K.
- 100 mm of fibreglass wool would have a u-value of 0.44 W/m2K.
Obviously walls/floors/roofs would be a bit better than just the raw insulation, but we are also ignoring thermal bridging and ventilation, so based on that taking a 2026 Swift mobile home with leisure spec we can see it is approximately the same as BS3632 residential spec between 1995 and 2004 – quite a bit worse for the roof, a bit better for the walls and floor. Which means a residential spec mobile home aged between twenty-two and thirty-one years old would be approximately equivalent to a brand new leisure spec Swift mobile home.
As mobile home parks force out fifteen to twenty year old mobile homes no matter their condition, there is a comparative surfeit of them, and they’re cheap enough e.g. a year 2004 residential spec Swift Moselle is currently on DoneDeal for β¬23k, and a 2010-2014 model can be had from the ‘Mobile homes all Ireland’ Facebook group for β¬26k. The problem with twenty year old mobile homes is you’re always going to be worried about structural rot or older appliances breaking down and I’m not at all sure how well I’ll be able to tell from inspection what works or not, or whether water ingresses when it rains etc. That said, I have seen residential spec mobile homes as recent as 2019 with a one year warranty from a main dealer for β¬32k. Still, as this is the cheapest bang for the buck, it is the most likely choice we’ll end up with.
Assuming of course that a decent one turns up on the second hand market after I have the hardstanding for it done. One advantage of buying new is you’re guaranteed it’ll be available and on your site by a certain date. If buying used, it’s far more luck of the draw.
Conclusion: A more than twenty year old residential spec mobile home is worth considering if and only if (a) it is in great condition and (b) it’s cheap enough. It is probably worth seeking one with a warranty from a main dealer, then at least somebody has tested the boiler and done a deep clean.
Depreciation and resale
As I alluded to above, mobile homes have steep depreciation curves – about the same as a car. You lose about one third of the value after three years, thereafter depreciation slows and they are usually worth about 50% of the current new price at ten years old. After fifteen years old because many mobile home parks force their sale, the lower spec ones tend to be plentiful and they take a fair price drop, and then because all mobile home parks force their sale after twenty years you get residential spec models aplenty. After fifteen years for leisure spec, and after twenty years for residential spec, they are generally worth 15-20% of the current new price. Some of that is because they can no longer be mounted in mobile home parks, so there are more sellers than buyers, and the price drops accordingly.
Who then buys all these old ex-park mobile homes? Given how quickly they clear from DoneDeal (an Irish online marketplace), it would seem there is a large market of families in exactly our situation: in the middle of building their house when they got evicted. So long, therefore, that the self building house building section of society remains strong then you should be able to sell any mobile home you purchase. And even if there were a hefty economic downturn, there would be a period where rents remain high and people get evicted and need somewhere to live. Only after rents had fallen due to lack of occupancy might the market for used mobile homes dry up.
All this stands in contrast to modular homes, which have become very popular in Ireland since a planning rule change this summer. These are generally wider than mobile homes and made from a steel frame rather than a timber frame, and they tend to be better insulated. They also arrive on the back of a truck and are craned into position. As they are wider, transporting them will be more expensive and relocating them after sale may be harder. There is also no existing data on what their depreciation rate will be, nor whether there will be a second hand market for them at all. I think therefore that these are probably best avoided for now unless you want to risk never being able to get rid of it in the future.
What’s next?
I think where we are at is probably we ideally want a residential spec mobile home manufactured after year 2005 (so max twenty-one years old), then you’re within the 2005-2014 BS3632 residential spec bracket. If you want it with a main dealer warranty i.e. somebody knowledgeable has checked it for leaks and broken appliances and is willing to stand over it, you’re probably talking β¬30-40k with a likely resale value of β¬20k on the used market after.
Assuming an eighteen month period to live in it, that is equivalent to a monthly rental of β¬833. If a twelve month period, then β¬1,250. Quite a bit more than we currently pay to rent this home (which is why we’ve remained here so long), but still way lower than any property currently being rented out anew.
Obviously this bombshell will rather upset my original plans for getting stuff done this Autumn. Most unfortunate. I now need to pack everything up in the rented house and move as much of its contents out into storage as possible, and that will take weeks to months and all that is time not spent doing what I had intended to do. Still, we were going to have to move out eventually in any case, so this is future work moved up to earlier than expected.
Speaking of said original plans … my custom AI harness is now dogfooding itself, so it now iterates rounds of review and fix upon itself and we are getting ever closer to something which has all its warts and corner cases ironed out. This has taken longer than expected: it fairness, only by dogfooding itself did we discover various architectural design mistakes and then when you change a bit of the architecture, that produces ripples which takes a while to tamp back down again. But we’re definitely into the last 5% of work on it. Then I intend to set it loose on multiple other projects, as now I’ll be safe in the knowledge it can be left run unattended and I no longer need to hover over it. That means I can get to bed earlier as I currently have to hover over it until 2 am each night as that’s when the peak rate time begins, then I set it running again at 11 am each morning as that’s when the peak rate time ends. Very much looking forward to not having to do that anymore.
Word count: 4583. Estimated reading time: 22 minutes.
- Summary:
- The use of multiple AI coding agents is described. Token costs and the Jevons’ Paradox are analysed. Three projects are detailed: ISO wording, a WebAssembly-based Python environment, and a secure container for unsupervised LLMs. Various subscription services are compared to minimise expenses. Finally, personal updates regarding construction and family are shared.
Friday 4 September 2026: 18:17.
- Summary:
- The use of multiple AI coding agents is described. Token costs and the Jevons’ Paradox are analysed. Three projects are detailed: ISO wording, a WebAssembly-based Python environment, and a secure container for unsupervised LLMs. Various subscription services are compared to minimise expenses. Finally, personal updates regarding construction and family are shared.
All three AI coding agents have been running Deepseek v4 Flash 0731 (DSv4F) which I reviewed in depth last entry and that has raised a new problem: cost.
Economics 101: Good Old Jevons’ Paradox
The Jevons’ Paradox is when technological improvements increase the efficiency of using a resource, when then paradoxically leads to a rise rather than a fall in the total consumption of that resource. This is due to elasticity of demand: when a thing becomes much more useful for a given cost, total consumption rises. This has been very much the case for me and LLMs during the month of August:
It’s actually even worse than the above: I burned a further two billion tokens on Opencode Go which is a US$10/month subscription – I ate the entire month’s allowance in three days which was the 23rd, 24th and 25th of August:
You can see on the 1st August my last use of Step 3.7 Flash, which had been the previous Pareto optimum, and that use was for the previous diary entry. Since then, it’s been almost nothing other than DSv4F.
I seem to get through at least 200 million tokens per day, with my worse case day so far being an astonishing 1.35 billion tokens when I had it go off and research ‘what would be the ideal LLM programming language?’ (the answer: Python, followed by C and quite some distance later, Rust – yes I was surprised too):
During the month of August, I reckon I paid about US$0.00865 per million tokens (it’s easier to say US$8.65 per billion tokens). That’s after Deepseek doubled their prices – they were half that cost again originally. As I’ve gotten better at minimising uncached token parsing which is the expensive part, I daresay the wholesale cached read token price of US$7 per billion tokens will get closer. Deepseek v4 Flash remains – for now – by far and away the Pareto optimum for my use case which is very heavy on cached read tokens: as of end of August 2026, only MiMo-v2.5 has cheaper cached read tokens at US$2.8 per billion, and it’s not clever enough to be left to run for hours without supervision (that said, I think there is a cost reduction strategy there which I’ll be coming back to later).
I’m not keen on spending US$100 per month on LLMs especially in my current zero-income circumstances, so I have been casting around looking for cheaper tokens. Deepseek themselves are unusual in that they offer no direct to consumer subscription, they appear to offer subscription pricing via third parties only. Of those third parties, here are the only places that I have found with cached read token pricing cheaper than wholesale pricing, ranked in order of bang for the buck determined via a thorough survey of the automatically updated pricings at https://codeberg.org/mutablecc/calculate-ai-cost/src/branch/main/subscription_vs_api_comparison.csv:
- Ollama Cloud gives you exactly 3x the monthly subscription price of US$20 in tokens, and therefore that is 3x cheaper than the DSv4F wholesale rate (and same goes for all the other models they provide). If you’re happy buying a full year of subscription, they knock 20% off the price, which makes this probably the cheapest source of DSv4F tokens at a minimum price of US$2.25 per billion tokens.
- Opencode Go gives you 1.5x-6x the monthly subscription price of US$10 in tokens depending on model. They list what multiple it is per model at https://opencode.ai/docs/go/#usage-limits. For DSv4F, it is currently 3x, same as for Ollama Cloud.
- ClinePass gives you 2x-5x cheaper than the wholesale rate for US$10 per month, and they don’t publicly document anything further. It would appear they are quite similar to Opencode Go in terms of bang for your buck in practice, but because they allow a dynamic rug pull any time they choose based on what suits them – and I have no idea what discount they offer on DSv4F tokens specifically – Opencode Go or Ollama Cloud are better choices as they publicly document what you get for your money.
- OpenRouter offers the public DSv4F wholesale rate of a minimum price US$7 per billion tokens, and it’s what you fall back onto when you’ve burned through your daily or weekly token allowances with the earlier three.
DevPass gives you exactly 3x cheaper tokens for a large range of backend providers the same as you’d get on Openrouter, but the monthly minimum subscription is higher at US$29 per month. There are two big issues:Maximum prompt caching requires special handling otherwise DevPass can cycle your session between backend providers, thus burning through your allowance much quicker.You don’t get to choose who provides DSv4F, they do, and they annoyingly don’t choose the provider with the cheapest cached read price so you end up burning through your US$87 much quicker than you would with the previous two options.
In fact, I ended up cancelling my subscription for this option, because the providers they chose ended up costing US$41 per billion tokens, which is 13x the cost of Opencode Go and 5x the cost of OpenRouter wholesale. So until they fix their provider selection, or better again let end users pin specific backend providers, I cannot recommend this service.
I am hardly alone in the same boat of seeking cheaper tokens wherever I can find them: this guy’s blog post is pretty much identical to what I just wrote, except he’s hunting for cheap Kimi K3 tokens. And you’ll find similar blog posts and Reddit posts etc all over the internet because there is a small army of us all in the same position.
Obviously your first start is the Ollama Cloud subscription for US$20 per month, giving you maybe six billion tokens at twenty percent below one third price (27.5% in fact). You might then add two Opencode Go subscriptions so they start their monthly cycle two weeks offset from each other to add an additional ~2.5 billion tokens at the one third price every second week. Any tokens needed above that you fill in using wholesale priced tokens from Openrouter. If I end up using fewer tokens in future, one can cancel either one of the Opencode Go subscriptions as needed, which is why I chose those and not 2x year long Ollama Cloud subscriptions.
To automatically switch between these providers, one would need some sort of dynamic router which programmatically selects between Deepseek v4 Flash 0731 providers based on rules, and thankfully there is a mature one of those called Omniroute. It, very usefully, already is Deepseek v4 Flash 0731 aware and automatically disables usage during peak hours in China and it also automatically can parse Opencode Go quotas so it doesn’t even need to poll whether the quota is fully consumed. Very nice!
We are now up to a minimum monthly spend of US$40 for around eleven billion tokens, which is less half the US$100 I spent last month on ten billion tokens. That obviously is not quite one third the cost, but as I mentioned I got better at not wasting tokens as last month progressed. We’ll see how September goes with the new arrangement.
Now, let’s get into the projects I’ve been having my concurrent instances of DSv4F working upon:
LLM Project 1: ISO normative wording
I am famously bad at writing ISO normative wording. Eight years full time on ISO committees and I still suck at it. But good news: LLMs are specifically trained to mimic style and form, and if you feed them the entire ISO C standard using a prompt like this:
You are an expert member of the ISO WG14 C programming language standards committee, with an eye for detail and a tendency towards nitpicking.
Read proposed-wording, then read the N3886 it links to (you may find a copy in the
referencedirectory). Ensure that proposed-wording applies cleanly to N3886. Where the proper tone, phrasing, terminology or vocabulary differ in proposed-wording from N3886, fix the proposed-wording.Read the abstract machine section of N3886. Is there anywhere in the proposed wording which ought to be rewritten in terms of the abstract machine?
Then comprehensively review proposed-wording to determine if an implementation could be faithfully made from the proposed-wording alone. Merge any missing detail or other issues into
plans/analysis.md. Try to makeplans/analysis.mda list of open action items.Make very sure you follow the style and conventions in N3886 such that after you are done, the proposed wording matches N3886 like a glove.
With that prompt, the LLM will turn my hand written wording such as this:
And the following macros, which restrict the optimizations which the compiler may perform:
sigfence(vars ...)for the following prevents the compiler relocating memory accesses from one side of the fence to the other side of the fence; it also causes the compiler to flush changes to memory bef ore the fence, and to reload from memory after the fence.
- the memory storing all values accessible by external or internal linkage.
- the memory storing values
vars ...without linkage.
sigfence()is async-signal-safe.
Into this:
7 At a call to the
sigfencemacro (7.14.1), the implementation shall not deviate from the abstract machine for the memory that the call describes: the value of the memory at the point of the call is the value most recently stored to the memory before the call; an access to the memory that is sequenced after the call reads from the memory; and no access to the memory is performed on the other side of the call from where it is sequenced.…
and the following macro, which restricts, for the memory described below, the freedom of an implementation to deviate from the abstract machine (5.2.2.4):
sigfence(...)provides the guarantees specified in 5.2.2.4 for the following memory:
- the memory storing all objects with external or internal linkage;
- the memory storing the objects without linkage named by the arguments of the invocation.
sigfence()is async-signal-safe. The behavior is undefined if thesigfencemacro is invoked with more than eight arguments. Each argument, if any, shall be an lvalue designating an object; the behavior is undefined if an argument is a bit-field or designates an object declared with theregisterstorage-class specifier. An argument that designates an object with external or internal linkage is permitted but has no effect, because the memory storing such objects is described by any invocation of this macro.
… which is far better than anything I have ever written myself – in fact, my own wording was me trying at 110% to write the best normative wording ever so far. As you can see, this just isn’t something I am good at, and now I have a normative wording writing helper. Yay!
Obviously I have been applying multiple rounds of analysis and refinement until the LLM could find nothing more. As I have four WG14 papers in flight right now, that took the month as one of my three concurrent projects, especially as you can also have the LLM repeatedly compare the proposed normative wording to the reference implementations, so everything moves together in lockstep and they all support one another. This is definitely not something I could do for a reasonable cost before DSv4F came out. I am very pleased.
LLM Project 2: Running full fat Python on real i386 Linux JIT transpiled into WebAssembly in the web browser
My son has a bunch of beginner Python teaching books, but they are all orientated
around desktop Python, and he only has a locked down Chromebook which can’t run
a desktop OS like Linux. You can get nowadays surprisingly good Python-within-a-webbrowser
implementations which offer almost all of a desktop Python experience, but the ‘almost all’
comes with a giant missing package: no tkinter package, because for obvious reasons
getting an X11 emulation running well on a HTML canvas is a big project and nobody
has done one yet. As a result, while the tutorial programs he types in for Python
turtle work great in Python-within-a-webbrowser, the lack of tkinter meant a good
chunk of those Python teaching books was wasted.
So I asked DSv4F to go off and research how to solve this (I had been thinking let’s get it to write an X11 implementation for HTML canvas) and it found an interesting open source project which is a virtual machine running real i386 Linux JIT transpiled into WebAssembly in the web browser. One could then – theoretically – boot into Python IDLE, and voilΓ there’s your full fat Python with all bells included but it runs within his locked down Google Chromebook.
That was the theory: it turned out to be considerably harder in practice, and indeed the LLM has been working on this all day and all night for four weeks to reach this:
(Note that for readers in years to come, I have rented the VPS which hosts that URL for one year, and I don’t intend to renew it after so the above iframe may now be broken. If that is the case, visit https://github.com/ned14/webbrowser-python-idle which may have a new live example link, or indeed you can just run the thing yourself locally from that github repo. Also, security policies may prevent WebAssembly loading within an iframe on your web browser, if so visit https://webvm.nedprod.com/ and that should work)
The reason that this took four weeks and probably a good US$60 of token spend is because it turns out that WebVM upstream is more a proof of concept code quality than anything actually useful. It is absolutely riddled with bugs, hangs, race conditions, missing functionality, and stuff that just doesn’t work at all even though it is supposed to and sometimes does if and only if the wind is blowing right. I really have to hand it to Deepseek v4 Flash 0731: it just kept at it, and at it, and at it. Night after night, day after day, until all known bugs were fixed, worked around, or otherwise mitigated. Even when we bumped the Linux and Python versions from the EOL versions used upstream and just about everything broke badly as a result, it just went at the iteration and drill down until it had it running again. I then had it invest five days in performance improvement, and we got boot times down from four minutes to fifteen seconds – though I’ll admit that I did a lot of helping here, because DSv4F would only ever take a very narrow view on optimisation and never asks ‘should we consider a completely different approach here?’. Which is fair enough: that’s not what this model is for, it’s meant as a worker bee not as the ideas person.
Still, I have to admit I admired its doggedness, and the fact it did eventually actually get there. And that inspired me into my third LLM project:
LLM Project 3: Enabling LLMs to be left to roam free unsupervised
As I mentioned at the top of this diary entry, there is a fair bit of involvement from me in supervising, reviewing and instructing the LLMs. That’s because they’re running in Visual Studio Code instances on my laptop, and because they can run any command at all, that means they could be doing anything. So I have to manually review every single thing they do several times per day, and this gets tedious very quickly. It also means that I have to leave my laptop permanently running and connected to the internet, and that that very much reduces the usefulness of a laptop.
So what if they could run free not on my laptop and I don’t need to care what commands they run? Then I could appoint a ‘manager LLM’ to review and manage ‘worker LLMs’, and I only have to do something when I get pinged.
To explain a little more, much of my current loop with LLMs is having it iterate prompts like this for some project:
Comprehensively analysis this project for correctness and implementation duplication, looking to de-duplicate where possible. Identify any missing test coverage which is low effort to add. Identify opportunities to improve performance during operation, and safety/good UX.
You set that running. It runs for a few hours. You manually review every command and thing it did and all the code it changed. You git commit, you run it again. Day, after day, after day.
There is zero reason why a manager LLM, probably GLM 5.3 which I’ll talk about later, couldn’t be launching and coordinating a bunch of DSv4F worker LLMs so I don’t have to. I might wire that then into say WhatsApp, and you could talk away with it including by voice note.
I am also very uncomfortable about anything which can run commands on any of my computers remotely. It’s basically a wide open security hole, and I am absolutely 100% confident that at some point somebody somewhere is going to take advantage of cloud LLMs to run any commands they like on people’s computers.
I don’t intend to be one of those victims, so before the LLMs turn nasty, my third LLM Project is to build a container for them so secure that I can legitimately ignore how they get the job done. For that, your container needs to meet these criteria:
Coding agents need to be able run any command they like, including package install, and it only affects an ephemeral filesystem within the container. Docker/Podman has exactly this, so it’s a good foundation to start from.
Each session should be named and runs in the background so I don’t need an open tty. Basically, I SSH in, create a session, set it going, detach it and it keeps going until I kill it.
Each session should have absolutely no visibility of the local network: it can’t even see the host NIC. All it has is a proxy for HTTP, HTTPS, and SSH for git, and those only work with explicitly whitelisted targets. Even DNS is proxied and only whitelisted items resolve. This prevents them going off and attacking some poor victim on the internet as part of a botnet.
One can mount parts of the filesystem of the host, but it’s keyed on physical button keypress (my SSH keyfob). The docker environment doesn’t do the mount itself, it asks a sibling container which does the auth, then that asks another sibling container to do the mount/unmount. Every container checks everything for correctness and prints by its own i/o to the human; also the sibling containers have 100% read only filesystems, so it should be fairly hard to mount anything not explicitly permitted by me.
It must not be possible to DDoS the host by consuming CPU, RAM, disc space or inodes. Docker has limits for CPU and RAM, but the LLM had to write code to enforce disc space and inode usage limits.
Pi.dev is the principle LLM harness running within each session. It does get a shared directory with all other pi.dev sessions so they can coordinate, and that shared directory does persist over time. One could checkout git repos into that shared directory, and LLMs could work on say refining and debugging a codebase over an extended period without worrying about losing data.
I haven’t finished this project yet as DSv4F has been beavering away at the security side of things for multiple weeks by iterating this prompt:
If an extremely malicious and very intelligent coding agent were running inside the pi-dev container, comprehensively analyse their potential ability to escape the container, damage data, or cause denial of service. Be extremely thorough. Leave no stone unturned. Go the extra mile. Analyse every possible corner case exhaustively.
As a result of those iterations, it has written such a large and comprehensive test suite for all the possible corner cases that iterating further changes takes a very long time. As LLMs don’t get tired, it just keeps going and going and going until it’s done. I’m also finding some of the ergonomics not quite right when I use it manually, so there is some design iteration left to do yet too. But I’m getting close, and then I get my laptop back to being a laptop once again. And I’ll be less anxious about leaving the laptop always running and god only knows what commands are being executed in my name as I sleep.
Managing LLM costs going forth
Obviously if I get the ‘Roam free LLM’ project up and running, I am likely to start burning a lot more than ten billion tokens per month. However, as I mentioned above, there may be scope to achieve this within my current subscription budget:
The manager LLM might run GLM 5.3, a 753b-40b MoE model which costs a minimum of US$86 per billion cached read tokens on Ollama Cloud. That’s expensive, but still way cheaper than any LLM of similar capability. It has an AA Intelligence Index v4.1.1 score of 60, which is at the time of writing the best possible score for any locally downloadable LLM.
(In case you’re wondering why not Kimi K3, OpenRouter thinks it costs 2x GLM 5.3 per task when used for agentic work. And yes, given the expense, we shall really need to keep the total tokens ever passing through this model as low as possible – it is 37x more expensive than DSv4F!)
‘High’ worker LLMs will run DSv4F, a 284b-13b MoE model which costs a minimum of US$2.33 per billion cached read tokens on Opencode Go. It has an AA Intelligence Index v4.1.1 score of 52.
‘Low’ worker LLMs will run MiMo-v2.5, a 310b-15b MoE model which costs a minimum of US$0.47 per billion cached read tokens on Opencode Go (about 5x cheaper than DSv4F). It has an AA Intelligence Index v4.1.1 score of 38. As mentioned above, it isn’t really clever enough to leave running unattended – it is very much a last generation model (three months ago hah!) – but it’s fine for simple very clearly specified jobs like reordering action item lists.
And of course prices will drop over the next few months as new models appear, and existing Pareto optimal models will have to discount, like MiMo v2.5 has. One then has the enviable choice between spending less for what we’ve already got, or maintaining the spend for better.
I suppose I should mention GLM 5.3 Flash, which OpenRouter thinks costs the same per task as DSv4F. A lot of people are excited about this just released LLM as it has an AA Intelligence Index v4.1.1 score of 57, so it looks better bang for the buck than DSv4F. However it is only similar cost because it’s currently on promotional pricing – next week it will double in price, and then it won’t look so good any more. For our cached read heavy use case, it doesn’t look compelling even at the promotional pricing, even if you assume it emits half the tokens of DSv4F: it would be more than twice as expensive, and not especially that much more capable. I actually unwittingly used GLM 5.3 Flash for about a week because there was an ‘unnamed frontier model’ going for free and with one trillion token usage caps. And no doubt, it was better than DSv4F overall, though I felt DSv4F was better in specific things like relentlessness during debugging. For totally free of cost, it was amazing, but at 2x the cost of DSv4F? Not worth it.
What’s next?
My builder informed me two weeks ago that he can no longer install my insulated foundations due to ‘lack of capacity’, so now I need to source an insulated foundations installer. Wonderful. I’ve gotten quotes in for all the materials, I’m just waiting for quotes for the labour. Hopefully I’ll have those by next week, and with a bit of luck we might get the foundations complete before 2027. Then I just need some walls!
Megan failed both her chartered accountancy final exams, so she’ll need to resit them in January and that means another four months of taking the children anywhere else so she gets time to study. That, combined with the insulated foundation works, means I will likely have little free time to spare. In any case, the employment situation has not improved – I am increasingly wondering if I ought to be thinking whole new lines of business rather than traditional tech consulting based around C/C++. The reality is that LLMs have partially nullified ‘the programming language question’ because devs no longer need to specialise in a particular programming language ecosystem. As an example of what I mean, that webvm project described above uses:
- 36% Javascript
- 22% Python
- 20% C (almost entirely diagnostic and quirks workaround patch code)
- 8% Shell
And the secure ‘Roam free LLM’ project:
- 51% Shell
- 41% JavaScript
I understand all of those languages well enough to competently generate a finished project, but I would have been much slower to work in those less familiar languages than I would be in say C, C++ or Python. Now the LLM can blat out whatever programming language, it may not be of ideally high quality as if I had written it, but for most applications that will matter far less than the human still needed in the loop to drive the final quality push before completion. Ultimately this is because LLMs don’t – and can’t – know what the finished project is supposed to look like, or how bug free it needs to be, or how quick its boot times need to become.
So meat bags shall continue to have a role for a while yet! But I think maybe future remote work might be more of a ‘get this problem solved’ type rather than a ‘we need a specialist’ type. I also can’t help repeatedly thinking that there must surely be future work in humanoid robots somehow, but I’ve not seen much happening in that area in Europe. Maybe things will look better in Q1 2027 after Megan hopefully passes her resits?
Until then, I have plenty to be doing. As normal after Megan’s birthday every year, I go teetotal until Christmas. This is usually my most productive period each year, this time last year I closed off multiple long standing todo items many of which had been open for years. Here’s hoping that this Autumn will also be a clanger for getting stuff done!
Word count: 18894. Estimated reading time: 89 minutes.
- Summary:
- A diary entry is provided. The release and performance of Deepseek v4 Flash is discussed. Comparisons are made with other LLMs through coding and summarisation tests. Predictions regarding AI evolution, future Apple hardware, and the impact of humanoid robots on employment are detailed. The diffusion of technology within Europe is also analysed.
Friday 7 August 2026: 00:01.
- Summary:
- A diary entry is provided. The release and performance of Deepseek v4 Flash is discussed. Comparisons are made with other LLMs through coding and summarisation tests. Predictions regarding AI evolution, future Apple hardware, and the impact of humanoid robots on employment are detailed. The diffusion of technology within Europe is also analysed.
Before we get into that though, yesterday my children finished painting the west wall white, thus concluding successfully the painting of around one hundred square metres of exterior wall. I think they did really great given their ages:
To complete a job like this over multiple days, it requires a focus and self control and willingness to see things through to when they are complete which I find lacking in most eighteen year olds, never mind much younger again. Well done Clara, Henry and Julia!
Cheap open weights AI leaps forward yet again!
You may remember that I was initially keen on Qwen3 Coder Next, it was rather slow on my ancient hardware but it did work. However I found myself thereafter mostly using Step 3.5 Flash rented from OpenRouter as it was surprisingly good at coding and agentic work, and it looks like I was early compared to most to realise this – however, then Step 3.7 Flash dropped, and it was better in every way however also twice as expensive for new input BUT now they had prompt caching implemented. Step 3.7 also emitted far fewer thinking tokens than Step 3.5, so all in all the actual cost paid dropped by about half, and I’ve found myself using Step 3.7 Flash for pretty much everything since its release as it had the best ‘bang for the buck’ from my testing i.e. Pareto optimum, and to be specific:
- It is not the most capable model by any means.
- It makes many mistakes in the code it writes.
- It can take multiple attempts to perform an edit or call a tool successfully.
- BUT if you apply multiple rounds of it checking its work it does catch 98% of the bugs and bad logic it writes and fixes them correctly.
- It is sufficiently cheap that I’ve spent a total of US$14 ever on it, and that’s despite it horsing through 160 million tokens …
- From my testing on my own actual use cases, it was the optimal cost-benefit choice of LLM for all tasks where the data it processed is public (I use a local small Gemma 4 LLM for anything processing data which isn’t already on the public internet)
- Things I really like about Step 3.7 Flash: it follows instructions well, it’s very hard to jail break it out of its system prompt, if you order it to be biased or non biased in its system prompt it does as it is told, and a 196b model is feasibly likely to be runnable on consumer hardware arriving soon, so it’s worth investing into mastering this class of LLMs as your daily driver.
Amazingly, it was only five months ago that Qwen3 Coder Next (Q3CN) landed; and just three months since Step 3.7 Flash landed. Now we have the final release of Deepseek v4 Flash, and here are those LLMs compared so you can see why everybody including me is so excited by this particular LLM release and why social media (or at least my view of it) has been jammed with Deepseek v4 Flash 0731 posts for the past week:
| Qwen3 Coder Next | Step 3.5 Flash | Step 3.7 Flash | Deepseek v4 Flash 0731 | Claude Fable 5 | ||
|---|---|---|---|---|---|---|
| Released: | Feb 2026 | Feb 2026 | May 2026 | August 2026 | June 2026 | |
| MoE weights (total-active): | 80b-a3b | 196b-a11b | 196b-a11b | 284b-a13b | 6t-a400b | |
| Max input context: | 262k | 262k | 262k | 1M | 1M | |
| Typical Openrouter input cost after prompt caching: | $0.103/M | $0.100/M | $0.053/M | $0.030/M | $3.36/M | |
| Artificial Analysis Intelligence Index: | 21.1 | 26 | 30.3 | 49.9 | 59.9 | |
| Artificial Analysis Analysis Index: | 36.2 | ? | 39.6 | 69.1 | 76.5 | |
| Artificial Analysis Agentic Index: | 8.8 | ? | 21.5 | 45.7 | 52.8 | |
| AA Omniscience Accuracy: | 15.8% | 23.9% | 25.4% | 37.2% | 61.4% | |
| AA Omniscience Non-Hallucination Rate: | 9.1% | 14.8% | 15.6% | 15.6% | 45.1% | |
| SciCode: | 32.3% | 40.4% | 40.0% | 49.9% | 60.2% |
For comparison, I placed in the final column the current best performing LLM anywhere which is Claude Fable 5. It is 112x times more expensive than Deepseek v4 Flash 0731! Until now the Pareto optimum Step 3.7 performed about half as well as state of the art – now you have something 80% as capable and for nearly half the cost of the previous Pareto optimum.
Deepseek v4 Flash 0731 is a model which at 284 billion parameters is still within the realm of near-future consumer hardware: by 2028, as you’ll see later on in this diary entry, your standard new Apple Macbook Pro from 2028 onwards is expected will include similar compute and memory bandwidth to a 2017-era nVidia Volta AI accelerator board. That should run a model like Deepseek v4 Flash well at around one hundred tokens generated per second, and maybe four thousand tokens parsed per second. That’s a good bit faster than my rented edition has been, so that’s more than fast enough for serious usage.
Between now and then, and especially as the price per token has just halved again with Deepseek v4 Flash 0731 while the capabilities took another leap forwards, it makes the most sense to rent. After this AI investment bubble bursts, I fully expect prices for renting LLMs to crash spectacularly, almost to the point of free … which may make buying local LLM capable hardware a tough ask especially if somebody invents an end-to-end cryptographically secure LLM execution engine, which I’m sure is just a matter of time. However, if your new Apple Macbook Pro just comes bundled in for no extra cost the capability to run local LLMs in the hundred of billion parameter class, then that’s a different matter again.
So, this new LLM aces the synthetic benchmarks. How does it perform in real life?
Test 1: Summarising a 19th century will
Replicating this test from last January, I had it summarise the raw transcribed will of my ancestor Walter Irvine which is early 19th century legal English full of transcription errors. Even the ancient llama 3.1 8b doesn’t do terribly at this, but at the time Claude Sonnet 4.5 did a great job. How do Deepseek v4 Flash 0731 and Step 3.7 Flash perform?
| Cloud full fat Claude Sonnet 4.5 400b? 8 bit? | Step 3.7 Flash 198b 8 bit | Deepseek v4 Flash 0731 284b 4 bit | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| (from the previous post, it was said at the time it was nearly instant) | Tokens per second ~220.4 tokens/s Token count 5987 tokens Cost $0.0091834875 Duration 27.2s | Tokens per second ~178.6 tokens/s Token count 4347 tokens Cost $0.00182941902 Duration 24.3s | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
This is a comprehensive probate document from Edinburgh dated November 17, 1825, concerning the estate of Walter Irvine, Esquire, who died on January 7, 1824, at Luddington House, Egham, Surrey. He held properties in the Island of Tobago (West Indies), Luddington House in Surrey, and Denino in Fife, Scotland. Scottish Estate Inventory (Total: Β£5,258 6s 2d)I. Rents and Arrears Due (Β£4,569 13s 2Β½d):
II. Bond from John M. Harlaw of Kirkton: Β£245 5s 1d plus interest to January 1824 (Total: Β£262 15s 1Β½d) III. Bond by Road Trustees (Anstruther-Crail to St. Andrews): Β£150 principal plus interest (Total: Β£162 7s 4Β½d) IV. Bond by Lundenfoot Road Trustees: Β£50 with compounded interest from 1821-1824 (Total: Β£55 1s 2d) V. Proceeds from sale of wood: Β£33 10s VI. Balance due from Walter Dickson WS (Writer to the Signet): Β£174 16s 11d plus interest (Total: Β£174 18s 11d) Will Provisions (dated October 18, 1823)Primary Beneficiaries:Wife - Catherine Irvine (nΓ©e Gordon):
Three Legitimate Daughters (each receiving Β£35,000):
Natural Daughter - Betsey (of London):
Other Significant Bequests:Ann Stewart (sister-in-law and widow of Stewart):
Brother's Estate:
Nephews:
Nieces (Clara and Marian Irvine - daughters of Christopher William):
Jane Armstrong (formerly Jane Currie, wife of Francis Armstrong):
Mary (reputed daughter of a negro woman named Linda in Tobago):
Other Named Beneficiaries:
Continuing Annuities:
Trust Structure and Powers:Executors and Trustees:
Powers Granted to Trustees:
Specific Provisions for Children's Trusts:
West Indies Property:John Leith Dispute:
Plantation Management:
Scottish Property Requirements:Election Doctrine:
Residuary Estate:After all debts, legacies, and annuities paid, the residue divided equally among three legitimate daughters as tenants in common:
Elizabeth's share for her separate use with power to dispose by deed or will despite coverture. Probate Details:Proved: January 28, 1824, in Prerogative Court of Canterbury Initial Grant of Administration to:
Reserved Power for: John Hamilton, Christopher Irvine, and Thomas Bruce to apply later Witnesses to Will:
Codicil: Dated same day (October 18, 1823), adding Β£100 bequest to Susannah Ann Hamilton Scottish Confirmation: Presented November 17, 1825, before George Carshin Jr., Commissioner of Edinburgh, with deposition by W.R.K. Douglas confirming the inventory and will details. The document demonstrates extensive wealth with significant properties in Scotland, England, and the West Indies, careful provision for family members including an illegitimate daughter, and sophisticated trust arrangements typical of wealthy estates in the early 19th century. |
Document Date: 17 November 1825 (Edinburgh) 1. Inventory of Personal Estate (Scotland)Presented by executor William Robert Keith Douglas (WS) and signed by him and George Carfrae Jr. (Commissioner). The total value of the Scottish estate is Β£5,258β―/β―6β―/β―2 (Β£5,258.31).
The testator also owned property in England; letters of administration for that part were already taken out by the executors. 2. Deposition (25 October 1825)Executor W.β―R.β―K. Douglas swore that:
3. Probate
4. Last Will and Testament β Detailed Summary4.1 General Directions
4.2 Provision for Wife β Catherine Irvine
4.3 Remission of Debt
4.4 Bequests to Individuals
4.5 Trust for Natural Daughter Betsy
4.6 Main Trusts for Daughters Christina Charles and Catherine Grace
4.7 Investment and Management Powers of Trustees
4.8 Devise of Real Estate
4.9 Executors and Guardians
4.10 Election Regarding Scottish Estates
4.11 Covenant for Daughter Elizabethβs Marriage Settlement
4.12 Residuary Estate
4.13 Receipts and Discharges
4.14 Powers of Executors
4.15 Appointment of New Trustees
4.16 Indemnity of Trustees
4.17 Management Powers
4.18 Execution
5. Codicil (18 October 1823)
6. Affidavits and Probate DetailsHandwriting Affidavits (for the codicil)
Attesting Witness Affidavit
Grant of Probate
7. Key Figures at a Glance
This document is a comprehensive and intricate testament disposing of a substantial transβAtlantic estate, providing generously for the testatorβs wife, daughters (both legitimate and natural), other relatives, and servants, while establishing elaborate trusts to preserve wealth across generations. |
This document is a collection of probate records concerning the estate of Walter Irvine, an Esquire who held property in the Island of Tobago (West Indies), Luddington House in Surrey, and Denino in Fife, Scotland. He died at Luddington House on 7 January 1824. The documents date from between late 1823 and late 1825 and comprise an inventory of his Scottish personal estate, a sworn deposition, his last will and testament (dated 18 October 1823) with a codicil, supporting affidavits, and the formal grant of probate from the Prerogative Court of Canterbury. I. Inventory of the Personal Estate (Scotland)The inventory was presented in Edinburgh on 17 November 1825 by Mr. Walter Dickson, Writer to the Signet, and made out by William Robert Keith Douglas, Esquire, one of the Lords of the Admiralty and an executor under the will (which was proved in the Prerogative Court of Canterbury in 1824, under the Act 48 Geo III cap 140). I. Rents and Arrears of Rent due to the deceased
II. Balance of debt due by John Hay Harlaw, Esquire of Kirkton
III. Sum in bond by the Trustees of the Anstruther & Crail to St. Andrews road
IV. Sum lent to the Trustees of the Lauderfoot road
V. Proceeds of roupes of wood due at death: Β£33. 10. 0VI. Balance due by Walter Dickson WS on account current (31 Dec 1823)
Total value of the deceased's estate in Scotland: Β£5258. 6. 2
II. Deposition (Edinburgh, 25 October 1825)Before George Carfrae Junior, Commissioner appointed by the Commissaries of Edinburgh, William Robert Keith Douglas swore that:
III. The Last Will and Testament (18 October 1823)The will revokes all prior wills, codicils, and testamentary dispositions. It directs all just debts, funeral expenses, probate costs, and legacies to be paid promptly out of the personal estate, with the real estate charged for any deficiency. Provision for the Wife β Catherine Irvine
Remission of Debt
Bequest to Ann Stewart (sister-in-law)
Other Bequests
Trust for Natural Daughter "Betsy"
Bequest to Jane Armstrong (formerly Jane Currie)
Bequests to Nieces Clara and Marian Irvine
Bequest to Mary (reputed daughter of enslaved woman Linda)
Annuities to Others
Bequests to Nephews
Main Legacies for Daughters Christina and Catherine
Powers of Trustees (Maintenance and Advancement)
Investment Powers
Devise of Real Estate
Appointment of Executors and Guardians
Election Requirement for Scottish Estates
Covenant Regarding Daughter Elizabeth's Marriage Settlement
Residuary Estate Distribution
Receipts, Powers of Executors, New Trustees, Indemnity, Management
Execution and AttestationExecuted on 18 October 1823 at Luddington House, signed and sealed by Walter Irvine across 19 sheets, in the presence of witnesses:
IV. CodicilDated 18 October 1823, adds a bequest of Β£100 to Susanna Ann Hamilton, daughter of John Hamilton, Esquire. V. Affidavits and Grant of ProbateAffidavit (Codicil)
Affidavit (William Adams, subscribing witness)
Grant of Probate (28 January 1824)
VI. CertificateLondon, 12 November 1825 β William Robert Keith Douglas certifies that this is the probate of the will of the late Walter Irvine referred to in his 25 October 1825 deposition on the inventory. Key Points at a Glance
|
Scroll down to see more ...
The good news is that it was much harder for me to determine the strengths and weaknesses of each of the two models compared to Claude from last year: Claude Sonnet 4.5 definitely still wins by a long shot, but the gap is much smaller than it was for Qwen 3.0 30b. Something which piqued my interest is why the Chinese models are so keen on outputting blow-by-blow structure of the original document, and I wondered if it is an artefact of the Mixture of Experts (MoE) design. So I also tested Gemma 4 31b which is dense and Gemma 4 28b-a4b which is MoE, and indeed the same blow-by-blow structure appears for the latter. I guess that kinda makes sense? Incidentally, Gemma 4 31b did surprisingly poorly on this test, I had assumed it would beat Deepseek v4 Flash as the Gemma models are well known to be better at English language nuance than the Chinese models, and while yes it did very well at picking out the right essential points from the will, it didn’t pick enough of those essential points despite being told to be detailed. Maybe I needed to say ‘very detailed’? Don’t get me wrong, the quality of Gemma 4 31b’s output was good, but it was short and to the point as it were, and too much short and to the point in fact.
Re: our two models, I think Step 3.7 produces a better structured documents – it is keen on tables – and it is more terse than Deepseek v4 which gives too much irrelevant detail, plus it writes English better in my opinion: less fluff, more densely packed. Deepseek v4 on the other hand did cost one fifth the amount which Step 3.7 did, and it’s not that much worse. Still, Step 3.7 wins this test on quality of output, if you exclude Claude.
Claude from last year is much better at synthesising the document together e.g. it groups daughters together, it has realised one is illegitimate, it orders items in a reasonable priority for most human readers, and it has collapsed all the multiple sections from the original into the minimum possible set. The Chinese models, despite getting towards a similar 400 billion parameters of Claude from last year, have a way to go yet, assuming that they’ll ever get there as they have a MoE design.
Test 2: Analyse a code implementation of a specification and implementation plan
Last few weeks I have been working on atomic_wait() for C, which essentially
ports the same feature from C++ 20 into the next C standard – though we shall
be adding some additional APIs, as we don’t care much for the C++ API. Myself
and fellow committee member Jens Gustedt came up with a draft WG14
proposal paper over a number of weeks, then I iterated having Step 3.7 Flash write
a detailed implementation plan for a reference implementation using another
hand written reference implementation for a separate WG14 proposal as a template.
It did struggle a bit with writing out the plan, and I had to hand hold it a fair
bit, but we got there.
The single most important part of the plan file is probably this which describes when a proxy atomic must be used which is indexed via an internal hash table, or whether the atomic wait can be passed through to the platform specific API directly:
| Backend | 1 byte | 2 bytes | 4 bytes | 8 bytes | Hash table needed? |
|---|---|---|---|---|---|
Linux (FUTEX_WAIT/FUTEX_WAKE) |
β | β | β | β | For 1-2-byte and 8-byte; futex is 32-bit only (int *uaddr, int val) |
macOS (UL_COMPARE_AND_WAIT/UL_COMPARE_AND_WAIT64) |
β | β | β | β | For 1-2-byte, or sub-native-width types |
Windows (WaitOnAddress) |
β | β | β | β | Never β all operand sizes bypass |
FreeBSD (UMTX_OP_WAIT/UMTX_OP_WAKE) |
β | β | β | β | For 1-2-byte, or sub-native-width types; UMTX_OP_WAKE accepts a count parameter directly |
pthreads fallback (pthread_cond_wait) |
β | β | β | β | Always β no kernel tracker exists |
β = kernel primitive available; hash table is bypassed. β = no suitable kernel primitive; must use the user-space hash table.
So, the design’s essential points are:
- There are multiple implementation backends for each platform specific API.
- The public API is able to pass through to the kernel API directly for some or all atomic types depending on backend.
- For the remaining types, an internal hash table maps an atomic’s address in memory to its proxy atomic which IS compatible with the kernel API.
I asked Step 3.7 Flash to implement the reference library using the plan and proposal as guides. It replicated over the mildly changed parts of the hand written source template well enough, but really struggled with designing and writing the implementation. It came up with a poor architecture and lots of repeated and verbose code all of which was unnecessary. So I ripped out all of that, and hand wrote an implementation for Mac OS which has compile time replaceable backends for different platforms. I then asked Step 3.7 Flash to implement the backends for (i) Microsoft Windows (ii) Linux (iii) FreeBSD and (iv) generic pthreads using the Mac OS backend as a template.
Here it did better, though it messed up the calling protocol, swapped a few arguments around, got error handling wrong etc. All not great, but then Deepseek v4 Flash 0731 was announced, and I thought it might be more interesting to see how both models would analyse the current implementation. I asked both this prompt:
Perform a comprehensive review of the implementation, comparing it to the plan and proposal. Write out any deviations, problems, bugs or any other issue into docs/deepseek_v4_flash_0731_analysis.md. Be detailed, also be aware that there is likely a race condition in the implementation which is causing lost wakes. Try to identify the cause without debugging.
Step 3.7 Flash took about four minutes to complete its analysis, taking
fifteen steps and costing US$0.075 dollars to process 101k tokens and
output 23k tokens. Deepseek v4 Flash unfortunately took my ‘try’ literally,
and launched gdb as part of its analysis. So I refined the prompt just
for it:
Perform a comprehensive review of the implementation, comparing it to the plan and proposal. Write out any deviations, problems, bugs or any other issue into docs/deepseek_v4_flash_0731_analysis.md. Be detailed, also be aware that there is likely a race condition in the implementation which is causing lost wakes. Identify the cause without debugging i.e. do NOT run gdb, do NOT run test programs - inspect and analyse the code ONLY.
… which worked, and it took ten minutes to complete its analysis, taking sixteen steeps and costing US$0.029 dollars to process 81k tokens and output 39k tokens. That’s about 2.6x cheaper, incidentally.
Its analysis was deeper, but it missed some easy issues Step 3.7 caught:
| Step 3.7 Flash 198b 8 bit | Deepseek v4 Flash 0731 284b 4 bit | |||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Step 3.7 Implementation Analysis1. Deviations from Plan/Proposal1.1 Windows 1-byte and 2-byte operations use hash table fallbackPlan: "Windows: always bypass (all sizes 1β8 bytes). The Windows backend never uses the hash table." Actual: Files: 1.2 macOS
|
| Issue | Severity | Location |
|---|---|---|
Hash table proxy stuck at atomic=1 after notify |
Critical | atomic_wait_common.ipp.ipp:220-222 |
| Lost-wake race: notify before park in hash table path | Critical | atomic_wait_common.ipp.ipp:41-45 |
atomic_wait_generic returns -1 instead of looping on proxy error |
High | atomic_wait_common.ipp.ipp:411-418 |
| Lost-wake race: notify without store in native path | High | atomic_wait_linux.c.ipp:43-44 |
Windows notify_all uses WakeByAddressSingle |
Medium | atomic_wait_windows.c.ipp:100 |
| Windows 1/2-byte ops use hash table instead of bypass | Medium | atomic_wait_common.ipp.ipp:524-528 |
wait_on_address32 discards errno |
Medium | atomic_wait_linux.c.ipp:50-51 |
macOS notify_all ignores max_threads_to_wake limit |
Medium | atomic_wait_macos.c.ipp:99-106 |
| No tests for hash table path | High | test/atomic_wait_test.c |
| No tests for notify-without-store | High | test/atomic_notify_test.c |
The most severe issue is the hash table proxy state machine (section 2.1 / race 3.1). The proxy is a single-slot binary semaphore that is set to "notified" (1) by the notifier but never reset to "waiting" (0) after consumption. This makes the hash table path fundamentally broken for any object that receives more than one notify, or for any waiter that arrives after a notify has already been issued for that object.
Deepseek v4 Flash (0731) β Implementation Review
Date: 2026-08-01
Scope: Reviewed the wg14_atomic_waits implementation against
docs/proposal.md (the authoritative spec) and plans/wg14_atomic_waits-reference-library.md.
Method: Static inspection of every include/ backend, src/ wrapper, the public
header, config.h, the tests, the build files, and CI. No debugger was run and no
test program was executed; all conclusions come from reading the code.
1. Executive summary
The implementation is structurally faithful to the proposal (all six public APIs exist, width dispatch for 1/2/4/8-byte atomics, native wait/notify backends with a hash-table fallback), but it contains one primary correctness race that produces lost wakes (Section 2), a broken pthreads backend (Section 3), a timeout smuggled in as an error on Linux (Section 4), a Windows lost-wake for multi-thread wakeups (Section 5), plus a number of smaller deviations from the plan/proposal.
The single most important finding: the per-object "proxy" used by the hash-table path sets a 0/1 notification flag that is never cleared while the wait-queue node is alive. This is the lost-wake/livelock race the task asked to identify.
2. PRIMARY RACE: the proxy notification flag is never reset β lost wakes / busy-spin
Location
include/wg14_atomic_waits/detail/impl/atomic_wait_common.ipp.ipp:
..._WAITmacro (lines 37β38):wait_on_address32(&(x)->atomic, 0, (timeout)) /* wait while proxy->atomic == 0 */..._WAKEmacro (lines 39β45):atomic_store_explicit(&(x)->atomic, 1, release), /* mono-directional: 0 -> 1 only */ wake_by_address32(&(x)->atomic, max_threads_to_wake)atomic_wait_generic()(lines 327β435) β the shared "park by proxy" loop.atomic_notify_generic()(lines 437β456) β the shared "set flag + wake" path.
The bug
A waiter parks by calling WAIT(item,...) which blocks while item->atomic == 0
(i.e. FUTEX_WAIT(&item->atomic, 0)). A notifier sets item->atomic = 1 and wakes.
There is no code anywhere that ever writes item->atomic back to 0 while the
wait-queue node is alive. The only place it is reset is at node creation inside
hash_table_find_or_create() (lines 220β222), which happens only when a brand-new
proxy_waiter_t is allocated. A node is freed only when use_count drops to zero.
Consequence β two interleaving outcomes
(a) Re-park after a wake never sleeps (livelock). Once any notify has fired on a
node, item->atomic is stuck at 1 for as long as the node lives. Any waiter that is
woken and must re-park β the proposal explicitly requires re-park on spurious wake, and
the code implements it as the top of the loop β calls
FUTEX_WAIT(&item->atomic, 0) while the value is already 1. The kernel compares
1 != 0 and returns EAGAIN immediately. Every subsequent iteration of the loop
returns immediately, so the waiter never sleeps again; it degenerates into a tight
100%-CPU busy-spin for the whole remaining lifetime of that node.
(b) Wakeups are lost because there is no sleeping thread. Because (a) means waiters
stop sleeping, a later genuine producer store + notify_* sets the flag (already 1)
and issues FUTEX_WAKE, which has nothing asleep to wake. The notify is therefore
effectively lost for the purpose of the sleep/wake contract; correctness then depends
entirely on the busy-spin poll observing the value change, which is not the semantics the
proposal defines and not what a correct reference implementation should do.
Why the analogous futex idiom would be safe but this one is not
The correct pattern guards the "am I allowed to sleep" decision on the same state that the notifier flips, and the notifier re-arms the state before waiting:
- waiter:
s = counter.load(); if (value == expected) futex_wait(&counter, s); - notifier:
counter++; futex_wake(...)Herecounteris a strictly increasing generation so the waiter can always detect a change that happened between its load and its sleep. The implementation instead uses a single 0/1 flag that is never re-armed, so the invariant "atomic == 0β a notify is pending/expected" is destroyed after the first notify.
Which configurations suffer
This path is the fallback for every backend whenever the operand cannot be handled directly by the kernel primitive, i.e. exactly the cases the proposal/plan force through the hash table:
- Linux: 1-, 2- and 8-byte operands (
HAVE_WAIT_ON_ADDRESS_*is 32-bit only). - macOS / FreeBSD: sub-native widths (1/2-byte).
- pthreads backend: every operand (there is no kernel per-address waiter).
The 4-byte Linux/macOS/FreeBSD/Windows fast paths and the atomic_wait_expected
native-width path bypass the proxy and are not affected by this flag, but the
8-byte-on-Linux case β a perfectly legal and likely test target β is affected.
Recommended fix direction
Replace the 0/1 flag with a monotonically increasing sequence number that the waiter reads before parking and passes as the futex compare value, and that the notifier increments before waking. Reset-on-rearm must happen on the waiter side before the sleep decision, under the same lock used to re-check the object value (or rely on the kernel re-check for the object value itself as the futex fast path already does).
3. pthreads backend is fundamentally broken (hangs / lost wake)
include/wg14_atomic_waits/detail/impl/atomic_wait_pthreads.c.ipp:
..._WAIT(lines 45β46) ispthread_cond_wait(&(x)->atomic, pthreads_mutex()).pthreads_mutex()(lines 61β73) returns a_Thread_localmutex, i.e. a different mutex object per thread.
Problems:
pthread_cond_waitrequires the passed mutex to be held by the calling thread. Inatomic_wait_genericthe waiter has released the hash-table lock (line 387) and then enterspthread_cond_waitwith a mutex that is never locked. This is undefined behavior; on glibc it typically fails immediately (EPERM) so the wait βsucceedsβ without ever blocking β again a busy-loop β and there is no guarantee the node is protected.- The broadcast hand-off is not protected by the mutex the waiter sleeps on. A
notifier holds the global hash-table lock and calls
pthread_cond_signal(via the..._WAKEmacro, lines 47β53). The classic lost wake occurs when the notifier signals between the waiterβs re-check (value still equal toexpected, line 380) and itspthread_cond_wait: the signal is dropped and the waiter blocks forever. With a futex, the kernelβs value re-check/EAGAIN saves this; withpthread_cond_tthere is no such guard and there is no predicate/flag protecting the check, so the wait is a genuine, permanent lost wake (a hang). - Even the
INIT/DESTROYmacros treatpthread_cond_tthrough the genericproxy_waiter_t.atomicslot, but the sharedatomic_wait_genericstill performs flag-style logic (settinguse_count, etc.) that is meaningless for a condvar.
Because CI runs ALWAYS_USE_PTHREADS_BACKEND=ON on Linux and macOS, this path is
exercised, but its crashes/hangs are exactly the class of lost-wake bug being reported.
4. atomic_wait_expected mis-reports a timeout as an error on Linux
include/wg14_atomic_waits/detail/impl/atomic_wait_common.ipp.ipp,
atomic_wait_expected_32() (lines 595β666), plus the Linux
wait_on_address32() (atomic_wait_linux.c.ipp lines 37β52).
- The Linux
wait_on_address32returns0on success/EAGAIN/EINTR and-1on any other error, not-errno. A genuine time-out ofFUTEX_WAITtherefore comes back as-1(witherrno == ETIMEDOUT). - The callerβs error branch (lines 651β659):
if(ret2 < 0) { if(duration != NULL && ret2 != ETIME && ret2 != ETIMEDOUT) { errno = -ret2; return -1; } }ret2is-1, which is never equal to the positiveETIME/ETIMEDOUTconstants, so the condition is always true for anyret2 < 0when a duration was supplied. A clean time-out returns-1(error), not0(time-out) as the proposal requires:Returns: β¦ returns zero β¦ or duration timeout occurs.
This is timing-dependent β if the pre-wait clock_gettime check (lines 631β637) happens
to notice expiry first it returns 0 cleanly β so the 1 ns test in
atomic_wait_expected_test.c is flaky, but the underlying error path is wrong.
5. Windows wake_by_address* only ever wakes a single thread β lost wake
include/wg14_atomic_waits/detail/impl/atomic_wait_windows.c.ipp,
wake_by_address32/wake_by_address64 (lines 92β124):
if(WakeByAddressSingle((PVOID)(uintptr_t) object)) return (max_threads_to_wake == 1) ? 1 : 1;
return 0;
- The
max_threads_to_wakeparameter is ignored; bothatomic_notify_allandatomic_notify(..., max_threads_to_wake=N>1, ...)call this and wake exactly one thread viaWakeByAddressSingle. The correct routine formax != 1isWakeByAddressAll. Every other waiting thread is left parked β lost wake. - The
(max_threads_to_wake == 1) ? 1 : 1ternary is dead code (both branches are1).
This makes the Windows backend incorrect for atomic_notify_all and for
atomic_notify with max_threads_to_wake > 1, which the plan marks as mandatory
behaviour.
6. macOS timeout conversion deviates from the plan
atomic_wait_macos.c.ipp, wait_on_address32/64 (lines 55β68, 77β90):
- The plan (Step 11) requires:
*durationβ nanoseconds, cap eachulock_waitcall atUINT32_MAX(~4.29 s) and loop for longer durations. - The implementation instead converts once to microseconds
(
tv_sec * 1000000U + tv_nsec / 1000U) and passes it in a single call with no cap and no loop. For any duration β₯ ~4295 s the microsecond value overflowsuint32_t, and durations beyond ~4.29 s are not split across multiple calls, so the accumulated wait can be far shorter than*durationβ violating the proposalβs βtotal accumulated time β¦ shall be at least*durationβ. - The code also declares private
extern __ulock_wait/__ulock_wakeinstead of including<bsd/sys/ulock.h>as the plan directs; functional risk if SDK/version behaviour differs.
7. FreeBSD 8-byte UMTX_OP_WAIT argument-order inconsistency
atomic_wait_freebsd.c.ipp:
- 4-byte:
_umtx_op(object, UMTX_OP_WAIT_UINT, expected, (long)&umtx_time)(lines 57β58) β passes the expected value in the value slot. - 8-byte:
_umtx_op(object, UMTX_OP_WAIT, (long)&umtx_time, (long)expected)(lines 88β89) β passes the timeout pointer in the value slot and the expected value in the address slot, i.e. the two are swapped relative to the 4-byte call.
The two calls are internally inconsistent, so at least one passes the operands in the
wrong order; UMTX_OP_WAIT (8-byte) is almost certainly wrong and will compare against
garbage / misbehave.
8. Return-value deviations from the plan
atomic_notify_32(lines 668β691) returns1 + reton a successful CAS, whereretis the number actually woken (0 on the proxy path when no node exists, 0 on the futex path when nothing is parked). So CAS-success-with-no-waiters returns1(positive), whereas the plan explicitly states:atomic_notifyβ¦ Returns 0 if the CAS fails or no waiters are parked.(The proposalβs βpossibly one plus the number wokenβ makes
1defensible, so this is a plan deviation, not a proposal violation β noted for completeness.)atomic_wait_genericsetsret = 1(line 388) before the first actual park call. The proposal ties βpositiveβ to βsuspended at least onceβ; the intent-to-park flag is acceptable but slightly loose.
9. Width-dispatch macros silently no-op on unsupported widths
atomic_wait.h, _WG14_ATOMIC_WAITS_IMPL_atomic_wait* / _notify* (lines 107β201):
- Each macro is
if (sizeof==1) β¦ else if (==2) β¦ else if (==4) β¦ else if (==8) β¦with no finalelse(anddo{}while(0)). A_Atomictype of any other width (e.g. 16 bytes, or along double, or a 0-width type) compiles to a silent no-op rather than a compile-time error, which can mask misuse. The plan states widths 1/2/4/8 are supported; a diagnostic would be safer.
10. Header-only / ODR notes
- The
atomic_wait_*_N/atomic_notify*_Ndefinitions in the.ippfiles are not themselves markedWG14_ATOMIC_WAITS_INLINE/static(only their prior declarations inatomic_wait.hcarryWG14_ATOMIC_WAITS_EXTERN, which isinlineonly whenWG14_ATOMIC_WAITS_ENABLE_HEADER_ONLY). In the multi-TU header-only build (header_only_test*.cpp) this relies on the C inlineextern-inlinemodel. It is workable on GCC/Clang but fragile; the plan required every backend helper to be explicitlystatic inlineto guarantee ODR safety. hash_table()usesWG14_ATOMIC_WAITS_IGNORE_MULTIPLE_DEFINITIONS(weak), so the singleton table is fine across TUs; this part is OK.
11. Smaller issues
errnonot preserved on success/failure paths ofatomic_wait_expected_32: it writeserrno = ETIMEDOUT(line 636) anderrno = -ret2(line 656), clobbering the callerβserrnoeven thougherrnopreservation was a stated plan requirement (Steps 8β12). The futexwait_on_address32does restoreerrno, but the caller overwrites it afterwards.EINTRtreated as success (return 0) inwait_on_address32(Linux line 45) rather than retrying; a signal during a timed wait silently shortens it.atomic_notify_genericdrops a notify when no node is registered (find_or_create withincrement_use_count=false, lines 444β445). For the futex backends this is masked by the waiterβs under-lock re-check; for the pthreads backend it is not masked (see Section 3), which is where the drop becomes a hard lost wake.- Hash-table growth/deletion re-probes correctly (cluster-shift deletion, lines 271β
280), but
hash_functruncates a 64-bit pointer to 32-bit before mixing (lines 108β113); not a correctness bug, just avoidable collisions.
12. Structure deviations from the plan (non-bug)
- The plan (Step 7) specified that
atomic_wait/atomic_wait_explicitbe thin wrappers delegating toatomic_wait_expected, andnotify_one/notify_alldelegate to awake_waitershelper. The implementation instead uses per-widthatomic_wait_1/2/4/8+ direct kernel calls for the native widths, with the hash-table path only for the fallback. This is a reasonable and arguably more correct choice given thatatomic_wait_expectedis native-width-only, but it is a deviation from the planβs described structure. benchmark_atomic_wait_test.cdoes not actually run a benchmark (returns 0 after declaring an unused consumer), so CI excludes it correctly but it validates nothing.
13. Conclusion
The implementation gets the public API shape, the width dispatch, and the native-width
futex path right, but the hash-table/fallback proxy logic has a one-way notification
flag that is never re-armed, which is the race responsible for lost wakes: waiters that
re-park on a node that has already been notified stop sleeping and busy-spin, and
subsequent notifications have no sleeping thread to wake. On the pthreads backend the same
logic is additionally broken by a thread-local mutex and a missing predicate under
pthread_cond_wait, producing hard lost-wake hangs. These are compounded by a Linux
timeoutβerror bug, a Windows single-wake bug for notify_all/max>1, a macOS timeout
conversion that violates the minimum-duration guarantee, and a likely FreeBSD 8-byte
argument swap. All were identified by code inspection only.
Scroll down to see more ...
Differences:
- Only Step 3.7 noticed that the Windows backend doesn’t bypass for all atomic sizes as it should according to the implementation plan.
- Step 3.7 false positives ‘atomic_notify_generic creates a proxy even when no waiters exist’.
- Step 3.7 side mentions the lack of a generational count: Deepseek v4 gets properly into the issue and describes it in full and does it well at that.
- Deepseek v4 has a much better handle on what’s broken in the pthreads backend and why.
- Only Deepseek v4 spots the inverted arguments in the FreeBSD backend.
Also, in general, the Deepseek v4 report reads more coherently and gets into the big issues quickly and tersely, whereas the Step 3.7 report is bitty and kinda all over the place.
Neither did a good job of identifying where the implementation plan or the implementation deviate from the proposal. The implementation plan specifically states at its top:
docs/proposal.mdis the authoritative spec. Behavior, return values, and memory-order semantics must match it exactly.
After this I stopped using Step 3.7 and exclusively used Deepseek v4. Perhaps the latter was overwhelmed by all the defects and deviations from specification with this first analysis which is why it didn’t perform well – all I can say is that later on, perhaps as the implementation’s major bugs got fixed which made logic analysis easier, Deepseek v4 Flash began to seriously impress me with its analysis capabilities. One still has to go through multiple rounds of something like:
Exhaustively compare the implementation to the proposal, looking for all cases of deviation, bug, issue, concerns and corner case where the implementation does not match the proposal, or the proposal may not be implemented in full. Be very detailed, thorough and methodical in your approach - go that extra mile each and every time. Analyse in depth all implementation backends and all combinations of build configuration options, also analyse in depth all error handling and control flow paths not normally taken. Update plans/combined-analysis with your findings.
… and then you rinse and repeat iterations of that, fixing one by one all the things it finds, and doing so until it finds nothing important in your opinion. So in that sense it’s like Step 3.7, but where it massively improves is on one shot fixes for CI failures: you simply tell it which CI failed and copy and paste the failure text. It then had a 100% success rate at finding and fixing the CI failure even for platforms it could not debug locally – it did so simply by inspection and analysis, including inspecting online the kernel sources for Linux, FreeBSD or ReactOS (to get an insight into Windows).
In fact, there was an especially impressive bit where it found that
Apple Clang 17.0 only would produce invalid binaries if symbol visibility
was set to hidden and a specific tail call optimisation caused a
function to be inlined into main(). It went off decompiled the Apple
Clang binary, compared it against the LLVM clang source code, found
the exact bit of problematic reverse compiled source code, wondered to
itself it it ought to patch the Apple Clang binary, and eventually decided to
instead hack around the problem for this specific instance and it even
added an informative explanatory comment to say why its hack was there.
Now, I’d read of Claude doing stuff like that. I’d seen Step 3.7 analyse
the assembler in binaries to figure out why they weren’t performing
as expected. But to actually localise a bug in a third party precompiled binary via reverse engineering?
That was new to me. No doubt it did take rather a long time to do all
that – due to Deepseek v4’s immense popularity right now, it has not
been running quickly, as little as 30 toks/sec. But I could leave it
chug away on its own safely I found, whereas Step 3.7 had a nasty
habit of occasionally wrecking your git repo or going off and installing
huge bits of software it didn’t really need via brew.
After the WG14 atomic waits reference library was finished, I had spent US$5.02 on 465 million tokens. That is US$0.0108 per million tokens. Yes that is an awful lot of tokens – in fact, I have consumed 913 million tokens ever on OpenRouter, so this one project consumed half my lifetime total – but the quality of implementation created is very high in my opinion. I would estimate it would have taken me over one hundred hours to create a similar quality implementation by hand – instead this cost me less than ten hours in total, which was almost entirely spent reviewing its work and giving direction on what to do next. Five dollars for ninety hours of my life back to do more interesting work is a bargain.
Test 3: Subjective experience of using each LLM to get work done
I think it’s fair to say I’ve been repeatedly wowed by Deepseek v4 Flash 0731’s capabilities. I HAVE found that you should not let it take architecture direction decisions: always ask it to present a menu of implementation options, and you’ll find half the time its recommended implementation is the wrong one. So that part sucks. But when you choose on its behalf the right implementation option, 98% of the time it does a great job: it matches the style and form of the existing codebase, it avoids writing copy and paste code and instead hoists common routines into reasonable locations in reasonable common header files, and the code quality written is well above most of the programmers I’ve ever worked with, with only very occasional slip ups. I really like the much improved one-shot fix capability, especially for platforms and architectures I can’t run on my system where I have the LLM agentic harness running. That’s been a HUGE timesaver: no more having to boot up Windows VMs etc to diagnose some random failure on CI.
I very much like its performance analysis. I asked it to make this codebase go faster. It spent some minutes pondering and reading code, and it told me we ought to use triangular probing instead of quadratic probing in the open addressed hash table as the buckets are a power of two, so the triangular probing would ensure better scattering of entries avoiding collisions. It one-shotted the new implementation, then benchmarked the difference, then twiddled a few unrelated items by parsing through the optimised disassembly as it knew my main ask was for improved performance. It then spat out hard benchmarks: 470 nanoseconds reduced to 50 nanoseconds. Impressive. It also generated comprehensive tests for scalability under load and that bucket growth did work perfectly under heavy multithreaded load. Even more impressive.
I asked it to add the Fil-C toolchain to the CI (this is a guaranteed memory safe C/C++ toolchain). It went off and found the documentation on the web, followed the instructions, set up the appropriate Github CI actions, adjusted the codebase where necessary as the Fil-C libc is musl rather than glibc, then to test it it installed a Linux VM as this is a Macbook, installed the correct AArch64 edition of Fil-C rather than the x64 one the CI uses, and ran the test suite via the local Linux VM under Fil-C. Worked first time in a single shot too – it didn’t make a single mistake. Had I done that by hand, I definitely would have made a mistake at least once – I know from past experience that setting up the Fil-C toolchain is finicky.
I originally had asked Step 3.7 to create this new reference library using an existing hand written reference library as its template. It didn’t do too well at that, and with hindsight I wish I’d have wiped what it did and started from scratch with Deepseek v4. Deepseek v4 Flash makes far fewer mistakes within the test harness (Kilo Code) and doesn’t need to self correct anything like as much. It also gets tool calling in the Kilo code harness right almost all of the time, unlike Step 3.7. I suspect it would have done a better job at mimicking the template into this new library, but I guess I’ll not find out until the next time I write a new reference library for WG14.
Deepseek v4 Flash is much more prone to proactively fix bugs and issues
without being explicitly told it can first. It seems to ask for forgiveness
rather than permission. So you’ll need to be careful to always git commit
before asking it a question, otherwise it may decide your question demands
code changes. At least it doesn’t like to git reset --hard
as Step 3.7 Flash was keen on doing when it got the codebase into
a confused state, rather it properly uses git stash so any working
tree changes can be recovered.
The 1M max context of Deepseek v4 Flash makes a BIG difference! I was used to beginning to sweat as the 260k context limit approached, trying to get it to write out todo lists into Markdown files for the next session clear as I always found context compaction just didn’t work well with Step 3.7. With Deepseek, I can just relax and let it trundle on – you are still wise to start a new session from time to time as the long contexts slow its execution down, but now you can take your time about it, and more importantly, if it goes off on a long extended think or ponder or diagnosis of something you can just ignore it because it won’t suddenly run out of context. This is the first occasion I can go do some other task while it runs in the background and I don’t need to stress constantly checking its progress. Very nice!
And finally, I really like how much cheaper it is. On Openrouter comparing usage and spend from before to after, I issued twice the requests and spent half as much money. I didn’t have to babysit this model as much as before. All in all, this is my new favourite Pareto cost-benefit optimum LLM choice. Well, at least for the next three months, if the pattern so far this year continues to hold true!
Which brings me onto …
Where LLMs and AI probably are going next
We now have enough history of LLM evolution to be able to predict with reasonable reliability where things will go next. As I mentioned above, I’ve chosen a new LLM coding assistant every three months on average this year. Shall I continue to do so?
I had Deepseek v4 Flash go off and scrape the AA Intelligence Index for a spread of LLMs over the past two years off https://artificialanalysis.ai/, then plot those using a contour map:
This has contour bands for ventiles in a LLMs AA Intelligence Index score and it shows that:
- For a <= 10 billion parameter model which is feasible for me to run locally given my ancient hardware, we broke into the tens around January this year, and we would expect to break into the twenties any time around now, followed by the thirties around January 2026, and the forties around Summer 2027. So, by Summer 2027, a 10 billion parameter model will score as well as Deepseek v4 Flash does. And it’ll run well even on this old Apple M3 based laptop.
- For a <= 300 billion parameter model which is likely to run well on near future Apple Macbook Pros, we broke into the twenties around last November, into the thirties last March, into the forties last week, and we would expect to break into the fifties before the end of 2026, then into the sixties by March 2027. Reminder: Claude Fable 5 scores sixty-one. So, before the end of Spring 2027, that six trillion parameter model will score similarly to a 300 billion parameter model!
Let’s graph time directly against AA Intelligence Index score:
What strikes you about this graph is firstly by how much the open weights models are catching up with the closed weights ones – I would be surprised therefore if the Chinese government continue to release their frontier models with downloadable weights in the near future. Secondly, there is a clear structural break between 100-200b and 200-500b models – there is wide space between their trend lines. Indeed, right now 200-500b is outperforming 500-1T, which is surprising. Less surprising is the 1T+ category which has a trend line matching that of the closed weight models.
The current most intelligent LLM anywhere by this index as of August 2026 is Claude Opus 5 with a score of sixty-one, followed by the current most intelligent free to download LLM which is Kimi K3 with a score of fifty-seven (Kimi K3 is a cool 1.4 Tb of download, and generally you need about as much VRAM as the download size to run it, so that would be very expensive to run locally given current RAM prices). Deepseek v4 Flash 0731 is 167 Gb to download, so it would run well on a machine with 256 Gb of VRAM, and it gets an intelligence score of 49.9. Of course, a single score is an average, and some models are strong and weak on specific domains compared to others. https://artificialanalysis.ai/ lets you compose comparisons of LLMs, so I chose these six as representative for this discussion:
Scroll down to see more ...
As you can see, the AA Intelligence Index score is made up of lots of separate indices, each of which is then weighted into an average overall score. On some specific domains e.g. GPQA Diamond or r3-Banking, already it’s a wash between recent models. On some others, there is a clear pattern of the gap rapidly narrowing, however there are always going to be things at which a five trillion parameter model will beat the pants off a 500 billion parameter model: tool use and logic aren’t those, but specialist knowledge and reasoning will be.
In other words, yes while the AA Intelligence Index score will improve over time for smaller models, that will be in those parts of the index which aren’t specialist knowledge and reasoning. Small models simply can’t store as much knowledge as larger ones, so I expect my estimations above to be rather optimistic i.e. the score improvements will be less for the smaller parameter models than one would currently predict by extrapolation from recent past. Especially because the AA Intelligence Index score is out of one hundred, so as models max out all parts except the specialist knowledge and reasoning, they will end up running into an upper bound where only more parameters can improve some of their domain specific scores.
ALSO all this prediction is contingent on the AI investment bubble continuing to inflate. One gets cleverer small models by investing more compute into training fewer parameters. For that, you need more and cheaper compute, and for that you need to keep investing those billions. This year they think ~US$900 billion has been invested in AI, and for next year we are on track for US$1.4 trillion dollars in 2027. Even the great wealth and income of the tech multinationals will struggle to fund so much debt – even now, total free cash flow for several of them is below their debt servicing costs for the debt they’ve taken out. But that’s another diary entry. In any case, it is hard to believe that the AI investment bubble won’t pop soon, and then we’ll have whatever compute has been built out by then and it’ll only grow linearly rather than exponentially after that, much like with the late 1990s telecommunications infrastructure investment bubble.
Linear compute growth does still enable model improvements, and unsurprisingly I’d expect them to stop improving exponentially and start improving linearly instead. As with the end of Moore’s law, the slowdown will affect the biggest highest end models first, and the smaller lower end models will see a long run of continuing exponential growth before that eventually also peters out. It’s been the same with CPUs: at the very cheap end, they’ve been continuing to exponentially improve the value per dollar cost for decades after the high end went into linear improvements. I think the same will apply to LLMs: after all, if training cost per dollar goes from exponential to linear improvement, the lessons learned from making the high end a little better should translate into larger improvements at lower ends, same as for CPUs.
I find this prediction of the future FAR more believable than predictions of imminent Technological Singularity which have started doing the rounds again. I covered that in the unpublished book I wrote after St. Andrews: the Singularity is purely the result of an artefact of human perception where we tend to weigh more recent big leaps forward as more important, as they are more important to us personally but aren’t really in the bigger picture of things outside humanity. Elon Musk had an interview with the Economist week before last where he was banging on about the Singularity. I suppose that suits his purposes to market that philosophy aggressively so fewer think about seizing some of his trillion dollars of personal wealth, but I also got the impression from the interview that he actually genuinely believes that a Singularity will happen at some point. I’ll categorically state right now: no Technological Singularity will happen in my lifetime unless some very new technology turns up. Certainly nothing about Large Language Models as presently designed and implemented is capable of generalised artificial intelligence i.e. AGI. Right now we’re in the exponential growth phase because we’re pouring exponential amounts of capital in – cut the constantly increasing capital investment flows and you can say good bye to exponential LLM capability improvements, as I just described above. All that said, the near term improvements to consumer hardware WILL be significant to Economic Total Factor Productivity as the gains from this technological advancement begin to diffuse widely throughout society.
Near future hardware
So that brings me onto the near future consumer hardware to run these things. Recent leaks say that Apple have started to design their next M-series and A-series chipsets to have better than the usual trendline of improvements to compute and memory bandwidth, so your 2028 Apple Macbook Pro should locally run < 500 bn parameter LLMs quite well indeed. Here are the current rumours and leaks in a single table and graph as created from the table by Deepseek v4 (at which it was surprisingly poor at doing interestingly, I really had to poke it hard and repeatedly to generate correct looking SVG, despite it amazing performance at graph building shown above – maybe the HTML table input upset it?):
| Model / Year | Edition | GPU Cores | Memory Bandwidth | Remarks |
|---|---|---|---|---|
| Apple M3 2023 | Pro | 18 cores | 154 Gb/sec | Unfortunately my personal Macbook is the M3 Pro, the worst for running LLMs of any of the Pro Macbooks π |
| Max | 40 cores | 410 Gb/sec | ||
| Apple M4 2024 | Pro | 20 cores | 273 Gb/sec | Added a memory cache shared between CPUs and GPUs like AMD's Infinity Cache for its GPUs. This greatly improved latency. |
| Max | 40 cores | 546 Gb/sec | ||
| Apple M5 2026 | Pro | 20 cores | 307 Gb/sec | First with hardware matrix multiply and accumulate (= nVidia 'tensor cores'). LLM input parsing is approx 4x faster than M4 as a result. 1024 FP16 FMAs per core per cycle enables 70 FP16 TFLOPs for the Max edition. |
| Max | 40 cores | 614 Gb/sec | ||
| Apple M6 2027? | Pro | 32? cores | 512? Gb/sec | Expected move to LPDDR6 standard memory architecture featuring a wider 24-bit channel layout (shifting to 384-bit Pro / 768-bit Max buses) to achieve a projected 1.67x generational leap in raw memory speeds. There will be no Max nor Ultra edition of the M6, this suggests that the core will be very similar to the M5 and they only upgrade the memory bandwidth. |
| Max | 64? cores | 1024? Gb/sec | ||
| Apple M7 2028? | Pro | 48? cores | 800? Gb/sec | Rumours say the Max variant can be fitted with up to 768 Gb of RAM in your standard Macbook laptop chassis. Obviously so much RAM will be VERY expensive as Apple likes to charge steeply for additional RAM. It would be surprising if TFLOPs don't double due to implementing 2048 FP16 FMAs per core per cycle. |
| Max | 96? cores | 1600? Gb/sec |
If you extrapolate out the numbers, the Apple M7 Pro should have the same memory bandwidth as a nVidia Volta enterprise AI accelerator from 2017, and the M7 Max should have the same memory bandwidth as a nVidia Ampere AI accelerator from year 2020. Chances are that the Macbook Pro and especially Max will have more VRAM (or equivalent, see below), but in terms of compute with 1024 FP16 FMAs per core per cycle they should pretty much match a Volta and Ampere exactly: the Volta maxed out at 125 FP16 TFLOPs and the Ampere 312 TFLOPs. Both had hardware matrix multiple and accumulate, same as the Apple M-series from the M5 onwards. As mentioned in the table above, it would be surprising if the M7 doesn’t implement at least 2048 FP16 FMAs per core per cycle given that today’s nVidia Rubin chipset can do 16384 FP16 FMAs per core per cycle, and Apple tends to follow closely whatever architecture choices nVidia makes – the M5 chipset’s GPU looks awfully like a nVidia GPU, just less wide. This architectural closeness is why LLM software support tends to be nVidia first, then Apple, then AMD (which is architecturally different), then Intel (which is architecturally different again). And why LLM software support on Apple is first class, whereas although support for AMD has improved enormously, it remains second class.
There are zero rumours about this next bit, so it’s probably wrong, but I would wonder if Apple would fit so much DRAM when flash mounted as Storage Class Memory (SCM) is (i) cheaper and especially (ii) much less drain on battery life. There is zero good reason why LLMs are stored in DRAM other than there isn’t an easily available cheaper substitute, but somebody big like Apple could simply fit NAND flash where the DRAM goes. You might only write that flash with an updated LLM every few months so its endurance won’t matter, and NAND flash if mounted like RAM is nearly as fast as DRAM. As the LLM model weights aren’t mutated in RAM, this could save easily 80% of the RAM demands of a LLM, so you get to run your 200 Gb sized LLM in 40 Gb of DRAM and probably less if you shrink the size of the KV cache which is very doable if you have Ampere levels of compute on tap.
Obviously that’s pure speculation, but I do know that DRAM is hard on battery life as it must be continually refreshed. Storage class memory would be easy to fit for somebody big like Apple and it would fix the battery life impact problem. I guess we’ll find out in 2028. In any case, you would expect parsing of around four thousand tokens per second, and generation of a hundred tokens per second on the M7 Pro – and double that for the M7 Max. That’s very acceptable for an ultrabook sized laptop.
Diffusion of local LLM capable hardware throughout society
Lots of ink both physically and virtually has been spilled lamenting how Europe isn’t keeping up with the US and China on AI advancement: we aren’t investing in the electricity supply for datacentres, nor in AI research past a small fraction of what the Americans and especially the Chinese are doing. It is therefore claimed that Europe will be left behind, and left at a significant disadvantage to the US and China.
This kind of claim has been made many times before on many topics of industrial, social and political comparison between the three superpowers – and it is true that especially recently Europe has felt on the back foot as it gets bullied simultaneously by the other two superpowers, which it isn’t used to historically. However, something less appreciated is that Europe is surprisingly good at diffusing more quickly and completely the gains of an advancement than the other two superpowers: it ‘buys in’ the advancement cheap, then mass disseminates it.
That will need explaining, so to simplify: Europe, due to its unique configuration of highly competitive constituent arms length states with huge size variations, tends to diffuse innovations faster and more broadly than America or China does. This is surprising on first inspection, but think of it this way: if Ireland obtains a large current account surplus by diffusing US sourced innovations widely across its economy, all cash strapped countries elsewhere in Europe start paying rapt attention and will try to duplicate and/or improve upon whatever Ireland is doing. Ireland gets a lot of stick internationally for being a tax haven and washing the profits of US multinationals of their tax obligations elsewhere – all of which is fair – but less appreciated is that all those US multinational operated subsidiaries in Ireland do genuinely diffuse US innovations throughout the Irish economy much quicker and more completely than they could in the US where they are nowhere near as relatively economically dominant. Same goes in Switzerland and Belgium incidentally.
Obviously I’m exaggerating a touch there – at times I do wonder about diffusion of best practices in Ireland – but my point is that in superpowers such as the US and China, practice of best practices tends to be concentrated in specific economic clusters such as New York or San Francisco-San Jose in the US, or Shenzhen-Guangzhou or Shanghai in China. Whereas Europe’s economic clusters are more geographically distributed and numerous in a unique three spoke configuration:
These are the famous blue, golden and green ‘bananas’ of European economic cluster (source). Unusually they all connect together through the North of Italy, which is exactly why while the Covid pandemic may have originated in China, it turned into a global pandemic in the North of Italy as that is the most connected place to other places in world bar none other, so all global pandemics will always spread worldwide from there. As with infectious diseases, so does the global diffusion and spread of new ideas and best practices all originate from Northern Italy.
And the same will undoubtedly apply to the mass adoption and use of LLMs: the US may design the hardware and the Chinese may manufacture the hardware, but it’ll be Europe who reaps the most economic value for the cheapest price from their inventions. This is why Europe always appears to be an economic laggard, yet by all metrics it has the best quality of life for the most people anywhere in the world despite having the lowest debt to GDP ratio of any of the world superpowers. Before some say ‘that’s because you don’t spend enough on defence’, I already debunked that in past posts here: Europe has rarely spent less than the US on a PPP adjusted basis, and last few years it is by far the biggest military spender in the world (and if you include Russia in Europe, which most would, then Europe has by far and away always spent more on its military than anywhere else in PPP terms). So, in terms of economic and welfare achievement, Europe’s practice of cheaply reaping from what others sow has served it very well.
How will this affect individual behaviours and mentality?
What will the world be like when your laptop and increasingly your phone locally runs a LLM as powerful or more powerful than the world’s currently most powerful LLM?
You might think what is different to the laptop or phone using a LLM running in a cloud elsewhere and using it over a data connection?, and in some ways you would be right: I’m using a Deepseek running in some cloud elsewhere over a network connection. What’s the difference between that and running it locally?
The first difference is privacy: I wouldn’t ever put anything potentially confidential anywhere near a public internet connection. I definitely wouldn’t put any personal emails or family photos near a public internet connection. Most people won’t care, so maybe this difference only matters to people like me. Still, I’m also an individual, and for me this matters a lot.
The second difference is cost centring: if a cloud runs the LLM, somebody has to pay for that and your average individual is highly adverse to subscriptions when a free of cost substitute is available. So 98% of individuals right now use the free LLM services, and they are generally terrible because otherwise they’d cost real money. If the LLM runs on your device, you take a hit to battery life, but otherwise it’s free of cost. So for your typical individual, from 2028 onwards they’re going to experience an enormous leap in LLM capability, as until then all they’ll be used to is the crappy cheap to operate free LLMs.
The third difference is that most businesses – and a fair few individuals – don’t like to introduce single points of failure to their operations. Most cloud services are seen by many as exceedingly annoying when they go down. And the more you depend on such a service, the more anxious you get if it could disappear/get cut off/drop out. If LLMs run exclusively on hardware you personally own and control, a lot of that anxiety lifts. Now you can lock yourself into this new technology with a certainty you couldn’t have had before. It is for this exact reason why private automobiles are so popular: you aren’t buying transport from A to B, rather you’re buying the guarantee of transport from A to B which public transport only offers in big cities.
The fourth difference is that if they’re truly free of cost and you can run them all day long and all it costs you is electricity, you’re going to use them a LOT more. For everything in fact. Why search the web if your local LLM can do it for you? Why order anything or reply to any message if your local LLM can do it for you? Why think about interacting with your device if your local LLM can do it for you?
And now we’re getting into the interesting stuff: what can a LLM automate away, and what can’t it do i.e. what role is left for humans?
We don’t still know how intelligent LLMs will become before the bubble pops, but I can say this: the LLM knows more about everything than you do, but not more about some specific topics than you do. Accepting on what topics you are weak but being honest about where you genuinely really do understand more than the LLM will be the key to your success going forth.
LLMs genuinely can be a force multiplier if you use them where their strengths lie, and combine that with your strengths. But they also hallucinate and are currently lousy at direction and strategy, so that’s where I would expect the value of humans to remain. In other words, I think politicians are going to have some of the best job security going forth, because their whole purpose is to set unpopular directions for everybody else.
How will this affect individual employment?
The future world of human employment I suspect is (a) those physical jobs which can’t economically be replaced by a robot controlled by a LLM and (b) those jobs where a human’s deep understanding of a niche topic of value cannot be surpassed by any LLM, or where decisions must be taken which involve long term direction and strategy. For everything else, I expect LLMs to gradually replace all before them.
Speaking of LLM controlled robots, I was quite surprised to discover that they only melded a LLM with a humanoid robot last June, so we’ve got a few years to go before humanoid robots start taking human physical jobs. But not as many years as you might think!
Much also to my surprise, it turns out that nobody was mass producing
non-toy humanoid robots until only November last year! Absolutely before
then as now you can buy toy humanoid robots,
these will dance for you
and do kung fu etc, but they’re absolutely useless for getting any real
work done as they (a) can’t lift enough weight reliably and safely and (b)
they don’t have the sensors for fine dexterity manipulation in unfamiliar situations.
And absolutely before now there were intelligent mass produced
industrial robots – any modern factory is stuffed with them – but none
were humanoid until last November.
The first mass produced industrial humanoid robot was the Ubtech Walker S2 which you can see to the right, and it went on sale in November 2025 and has probably sold about three thousand units. It costs about β¬150k ex VAT, it can carry up to 15 kg and you get about 2.5 hours per battery charge, though it can swap out its battery at a battery recharge station on its own so it can work continuously without a break. Its intended use is within pristine environments where fairly fixed programming works well e.g. walk over there, pick up one of X, rotate it until it has the right orientation, walk back here, put it into the right component box. In other words, just like any other industrial robot, but this one is capable of adapting to different locations within the same factory.
Next up is the Boston Dynamics Atlas which entered mass production in January 2026, and probably about two thousand units have been sold so far. It costs about β¬200k ex VAT, it can carry up to 30 kg including an impressive 20 kg if on one arm, and you get about two hours per battery charge. It seems a bit more intelligent than the Walker S2, but not by much: it is also intended for fixed, repetitive, work in a pristine environment like a factory floor. This robot is undoubtedly a lot more impressive in the build quality sense than the Walker S2, but it does cost a third more, and also Boston Dynamics only put it into mass production now after decades of development because they had to due to Chinese competition – not because it was finished or particularly compelling or priced well. It also is less interesting because all its production for the next two years is already sold to Hyundai, so nobody else will be able to buy one for several years more yet.
Last April, a much more interesting industrial humanoid robot went into mass production: the Figure 03. Here is a youtube live stream of it unpacking parcels in a mail office, ensuring that the address label points downwards for scanning:
Firstly, the handling of irregularly sized, sometimes squishy, items is FAR harder than the regularly sized boxes with grab handles that the previous two robots can handle. The Figure 03 can also climb stairs by itself – albeit slower than a very elderly person – but it does get there. It currently costs about β¬100k ex VAT, it can carry up to 20 kg and you get a very good five hours of battery life, but at the cost of it being 40% slower at movement e.g. it walks slower, moves slower etc. They have sold maybe four thousand of these by now. It comes with a bundled LLM running locally which isn’t particularly good – nowhere near even Deepseek v4 Flash in fluid conversation – but if you tell it to go wash the clothes in the washing basket it’ll go fetch the basket and take it to the washing machine, very slowly pick each item out and put it into the machine, then very slowly pour in detergent and set the washing machine running. Ultimately, apart from the thing getting in the way a lot due to its lethargy, it is a far more interesting humanoid robot – at least for the very wealthy, not least due to its cost, but also because you would really need a home with large open spaces so you can easily get around the robot while it very slowly does things. Just to be clear: the Figure 03 can run and jog as fast as a human, but it absolutely horses through its battery if it does, plus it gets hot – very hot! Still, maybe future firmware revisions could let it exchange battery for speed for short bursts so it isn’t annoying and doesn’t get in the way, but otherwise conserve battery life.
If I am being honest though, I suspect the Figure 04 is the one to wait for, as the Figure 03 feels like it has too many design and hardware compromises, and it is still too expensive for what you get on the software side. If it’s the most impressive in mass production right now, what screams out loudly is just how immature and unfinished its software story is. It’ll be years, at best, before that can be remedied.
In case you’re wondering what about all the other mass produced industrial humanoid robots, that’s it: everything else isn’t actually in mass production. In particular, Tesla’s very long advertised robot is nowhere to be seen: we don’t know its specs, its price, or anything else about it, and given Elon Musk’s long history of made up claims about autonomous driving, I wouldn’t be optimistic that his robot will have good autonomy for at least several years after launch. Ultimately this is because it’s one hard thing to build the hardware for an affordable price, it’s another hard thing to create compelling software for that hardware platform. As an example, Meta solved building affordable VR headset hardware, but they did not solve building a compelling software ecosystem for it, so the whole thing has gone off to die and it’s only a matter of time before that entire ecosystem is abandoned. Similarly, Tesla’s fully autonomous driving will likely never get solved well enough to be allowed by regulators at a price consumers will pay.
As much as Figure 03 is impressive, it still requires a pristine environment i.e. you can’t be taking it onto a building site. Even if a robot could navigate well such an irregular environment, and it coped well with getting mud and sand into its joints, it would almost certainly move too slowly for many tasks on a building site AND generally annoy the human construction workers by getting in the way.
Currently a construction worker might cost about β¬100k to the employer, so maybe for β¬100k a construction site robot might be worth the expense if it only did things like fetch concrete blocks so the blocklayers could keep working without pause. But as each block weighs 30 kg, it would need to be able to move wheelbarrows of them over scaffolding, which is far beyond the capabilities of any current or near future humanoid robot. You’d also need several of them as they’d go much slower than humans, and because they’d run out of batteries after a few hours you’d either need a quick battery swap facility or even more robots. And finally most construction sites don’t have electricity apart from a generator, so charging robots at a site would be very unattractive. So, for certainly the next decade, I think construction workers can rest in peace that they will not get made unemployed by humanoid robots.
For human jobs doing physical labour in pristine environments though, the next ten years looks like increasing levels of human jobs being displaced. If they can get the cost of these robots down to β¬25k, a lot of minimum wage jobs like stacking shelves or packing online orders look inevitably gone forever, as the minimum cost to the employer of a human (minimum wage is about β¬32k) makes the robot look cheaper. If somebody successfully cracks deep cleaning by robot, that’s all your cleaning staff gone too. Jobs like fast food kitchen work is at threat, even if the delivery driver is not – that’s a lot of your young person entry level jobs disappearing forever there.
The most recent (2024) ESRI report lists the largest number of minimum wage jobs being in these sectors:
- Kitchen helpers (14%)
- Shop sales assistants (10%)
- Bartenders (7%)
- Caretakers (6%)
- Waiters (6%)
- Home based personal care (3%)
- Housekeepers (2%)
- Receptionists (2%)
About ten percent of the Irish workforce earns near minimum wage, and humanoid robots could take over most if not all those eight sectors if they get cheap enough.
Food for thought indeed! This displacement of humans from their jobs by AI might have impacted IT first, but I am extremely sure it shall be coming for entire sectors of knowledge worker and pristine environment manual labour next.
Word count: 2666. Estimated reading time: 13 minutes.
- Summary:
- The as built survey is documented. A boundary wall was painted, and a water intrusion issue in the pool cavity was encountered during testing. Furthermore, reference levels were remounted, and the as-built survey data was compiled following the completion of groundworks.
Thursday 23 July 2026: 20:29.
- Summary:
- The as built survey is documented. A boundary wall was painted, and a water intrusion issue in the pool cavity was encountered during testing. Furthermore, reference levels were remounted, and the as-built survey data was compiled following the completion of groundworks.
That ended up being four coats of cheap stain and one coat of expensive stain: we probably should have applied a fifth coat of cheap stain to reduce the blotchiness by a bit more, but we were out of time as camps began the next day so I just had to draw a line under it. It is my no means bad, just not as evenly coloured as I would personally prefer. Clara and Henry did well, it was their first time painting anything big and given the hot weather, staying the course required will power. My son, aged nine, clearly was very done with painting by the end of the fifth day of it, but then he is aged nine. Both did good quality work, I only had minor touch ups to do and those were principally due to their lack of experience painting a wall. Total all in cost including paying them β¬150 between them was β¬300 inc VAT for twenty-four metres of wall, or β¬6.25 per sqm. That’s very good in my opinion for painting an unpainted wall which tends to soak up the paint first time.
The pool cavity becomes a pool!
An unpleasant surprise turned up part way through the left wall staining: the pool cavity became a pool! Here it is with the water already mostly pumped out:
The water I reckon was about 400 mm deep if you include the stone layer at the bottom, and about 330 mm deep from the top of stone within that pool cavity. Very obviously, the tree cavity – which is much deeper – was dry as a bone, and it has not rained to any extent in weeks. So it is a mystery how it filled up like that!
I took the opportunity to use the water to test my percolation test hole. While we were digging out the pool cavity, we found at the bottom some possibly wet soil whereas the tree cavity was dry as a bone all the way down. That got me curious, so I had the groundworks guys dig me as deep a hole as the digger would go right next to the pool cavity where the future soakaway was planned to get installed:
That’s about 1.6 metres deep, and you can see a little bit of ground water down there, so obviously the water table is somewhat high around here. My concern was that installing a soakaway two metres down would be pointless if it was below the water table in summer, and I can confirm that that little bit of water didn’t budge in the days after it was dug. When the pool cavity filled up, it still didn’t budge after several days, so I wondered what if I filled that percolation test hole and emptied the pool cavity. Here is the result after five days:
It may have dropped by an inch in five days during ~28 C constant sunshine weather, so I’m going to claim that the height loss was entirely evaporation. The pool cavity remained bone dry after it was pumped out, and as the top of water in the percolation hole is about half a metre above the bottom of the pool cavity, so clearly my subsoil has almost zero percolation. This was another unpleasant surprise, as I had specifically chosen this site expecting subsurface static water pressure to cause flow over my subsoil heat exchanger into the ditch just behind my property, making the heat exchanger more efficient. Apparently I have here the exact opposite π.
My current working theory is that my neighbour may have doused his garden with water to keep it alive, and due to the lack of percolation all that water promptly poured through his upper layer of topsoil over his subsoil into the pool cavity. If this is the case – I shall be watching things carefully – then there is a danger that the RC box going into the pool cavity could float if enough water surrounds it. To calculate that:
Pool cavity including EPS: 5885 x 3730.
EPS thickness: 200, with density 40 kg/m3.
RC thickness: 200 at bottom 215 at sides, with density 2,500 kg/m3.
Empty middle: 5055 x 2900, with density 1.25 kg/m3 (air).
So:
- Total horizontal area = 21.95 m2.
- Horizontal area of RC box = 18.27 m2.
- Horizontal area of empty air within RC box = 14.66 m2.
- Horizontal area of RC box = 18.27 m2.
- Bottommost 200 mm of EPS = 4.39 m3 => 175.60 kg.
- Nextmost 200 mm of RC box = 3.65 m3 => 9,133 kg plus 30 kg for the outer EPS => 9,163 kg.
- Remaining H metres of RC box = H x (40 x (21.95 - 18.27) + 2500 x (18.27 - 14.66) + 1.25 x 14.66)
- H x (40 x 0.736 + 2500 x 3.61 + 1.25 x 14.66)
- H x (29.44 + 9025 + 18.33)
- => H x 9073 kg.
Therefore:
H x 2195 (water) = -(9339 + (H - 0.4) x 9073) (RC tank)
H = ~0.51 metres above the base of the RC box
As the pool cavity is 0.77 metres below the top of T2 stone, and the T2 stone is about 200-250 mm deep, any water entering the pool cavity should overflow under the rest of the house before the RC tank would float, so we’re safe.
That said: you better believe I’ll be running the land drain out of the pool cavity no higher than 0.5 metres from its bottom, just in case.
Remounted the reference levels
Many years ago now, I had screwed three pieces of wood to the wooden fence to mark 75.08 over sea level, and all works since have used that as the reference level. Unfortunately, one bit of wood split and so had to be removed, and another got put out of alignment due to the wood staining. That left me with just one reference level marker, which got me anxious due to the lack of redundancy, so before the professional surveyor arrived I remounted two additional reference levels matching the one remaining one.
As my laser level is an indoor only model, I had to wait to dusk to be able to see the laser:
I remember being very, very tired when doing this work as I had been up at 4am and having to stay up until dusk really sucked. But I knew I’d get a lie in the following day, and I think I ended up asleep that following day until about 1pm as I had built up such a sleep debt.
When the surveyor came, we found that the bottom of those reference markers is more like 75.07 rather than 75.08. Thankfully, it didn’t seem to affect the stone levels according to the same survey, they were all spot on or slightly lower. I was also rather glad to learn that my marker on the wall other side of the site was just 4 mm off: my laser level, a Bosch prosumer model, promised <= 0.02% height error per metre of distance – over 35 metres, you would expect the error to therefore be less than 7 mm, so it was well within claimed specifications and only 43% worse than a commercial laser level.
The as-built survey
My surveyor, knowing that I have been more than a year without income, very kindly offered to reduce his bill if I did the post-processing work of taking his raw data and adjusting it to calculate the locations of all the popups. To explain, the problem with a pipe sticking up is you can’t directly measure the X-Y location of its centre, rather you can measure its side and then you need to offset that X-Y location in to the centre of the popup by half its diameter plus 15 mm (which is half the width of the X-Y location measuring stick). My surveyor would normally do that work and charge me for his time, but he reckoned me technically able enough to do it myself. And I am most grateful to him for that, every euro saved counts right now. Here is my as-built image of the site overlaid on the most recent drone photo taken a few days ago:
The levels shown above for the pool cavity, tree cavity and steel pad cavity are all as built; each popup as built is shown as a small green dot with its deviation from plans in text next to it; and the foulwater (FW) and stormwater (SA) all show the bottom of each pipe or inspection chamber as built. As with the sewer levels being somewhat off in places by up to 80 mm, so were the X-Y popup locations where the deviation was between 20 and 70 mm apart from the ESB ducts which are 100 mm off. Those ESB ducts being so far off is annoying, but as they are ESB ducts you can give them repeated kicks to get them back into position without disturbing the falls below in ways which matter (i.e. you can’t kick sewer pipes into position, you might create a trap) and/or you can bend them back into position as they go through the concrete slab.
So I think we’re all good, the only place where an out of place popup will matter is the one for the washing machine where it now pops up within the wall. We’ll have to perforate the wall, but as it’ll be behind the washing machine and you can’t see it I don’t think I care.
What’s next?
Here is the site right now. It’s only taken more than five years and β¬440,000 of my money to reach this:
Utter madness really. If I could go back to when I started this knowing what I know now, I would not have started building this house. It isn’t worth the years of time nor the hundreds of thousands of money I could have spent elsewhere.
Still, we are committed now, walking away would consign to the dustbin most of the money spent, so we can only go forwards.
Unfortunately, the remainder of the summer won’t be much fun for anybody:
- Next week the kids will most likely be painting the rightmost concrete block wall white. I’ll give them break days in between to decompress, but I expect it’ll take the week to complete.
- The week after that, the first week of August, has a bank holiday and I might take them somewhere nice. Megan has been studying all day long every day these past two weeks and shall be for the next two weeks, so I just need to take the kids away elsewhere. We do also have some furniture to upcycle, it just needs to be sanded and painted, it shouldn’t take long and most of the structural reworking has already been done.
- The second week of August is Megan’s final chartered accountancy exams week, that will surely be a stressful week for all.
- The third week of August is a WG14 C standards meeting, at which I’ll be presenting three of my papers. I’m sure Megan will take the kids to fun places as she’ll likely want to go have some fun herself after all these years of study.
- The fourth week of August I believe the kids return to school at the end of it, so the summer will be over for them.
You can see in the photo above that the soil spill pile hasn’t been removed! We’re waiting in Green Valley’s queue to have that removed. As with the groundworks, we won’t know when they’re going to remove it until the day before, so until then we’ll be unable to book anything much in advance lest it get wasted. So I expect e-bike rides, maybe day trips to things nearby which don’t cost much e.g. beaches, West Cork, that sort of thing.
What will come after August? That’s a very good question. I have taxes to sort out, we’re in a novel situation I haven’t dealt with before with those, so they’ll need to be figured out and that’ll likely take a few weeks and a few grand of fees to advisors. We are currently likely to run very low on cash some time this Autumn according to current projections, so I’ll need to do something to get ahead of that before it becomes critical.
Much I guess depends on when this AI investment bubble bursts. I was reading this week that at least 1.6 trillion dollars of AI datacentre building debt has been accumulated into off balance sheet Special Purpose Vehicles (SPVs), which is the same mechanism Enron used to blow itself up in the early 2000s. Apparently about half the borrowing comes from pension funds, a quarter from wealthy individuals, and the remainder is circularity of cross investment by SPV owners and government subsidies. When the party stops – and it will – contagion will set off a chain reaction of SPV collapse and at least one tech multinational will likely see a run on its stock, which means it’ll need to be bailed out by the taxpayer or bought at firesale prices by a competitor. So just like the bank bailouts and collapses of the 2009 financial collapse, but this time with tech multinationals. This fun game of musical chairs guarantees some will cease to exist, and the theory is that that whoever remains will thereafter profit handsomely.
I’m not personally sure that’s the case at all as I think those who build out the infrastructure won’t have the moat to reap more than a tiny slice of the profits from it – we saw something very similar with the telecommunications bubble up to the 2001 tech bubble collapse where most of the benefits and value accrued to those who used the telecommunications infrastructure, and very little to those who had built it out. If true, places like Europe which have mostly ignored the building out part of AI will benefit handsomely having skipped paying for its development, and getting to deploy upon it with all gusto as their powder will be still dry. Maybe there’s a diary entry in there explaining my thoughts on that, maybe I’ll write that in the Autumn if I run out of other things to do. We’ll see.
If time presents, I really do need to get back onto open source library development as I need to refresh my portfolio so I look more employable after the AI investment bubble bursts. It would do no harm for me to present at a few conferences either – I did submit to ACCU this year, but my talk was rejected. I’ll try again for 2027. Hopefully, though, I’ll have a house on my site soon and I can go work in that each and every day until we can move into it.
Word count: 6185. Estimated reading time: 30 minutes.
- Summary:
- The installation of the third round of groundworks is documented. Extended hours were spent on site, during which a significant underground cave was encountered and bridged. All foulwater pipes and inspection chambers were installed, pressure tested, and levels were accurately recorded. The project’s substantial costs are also detailed herein.
Friday 3 July 2026: 21:28.
- Summary:
- The installation of the third round of groundworks is documented. Extended hours were spent on site, during which a significant underground cave was encountered and bridged. All foulwater pipes and inspection chambers were installed, pressure tested, and levels were accurately recorded. The project’s substantial costs are also detailed herein.
Most of each day I press on with chores around the site or on this laptop (e.g. writing this very post, which I began writing about two weeks ago!), though this past week I’ve had the kids onsite as their school has ended for the summer which has severely reduced my productivity. Several times a day I get called upon to take decisions, do layout or interpret engineer or architect drawings. Most of the decisions are easy, they just need to be explained so everybody is on the same page, but some are genuinely hard with no obvious answer. I, as the eventual home owner, the buck stops with me so you take your best guess and we plough on.
In case you’re wondering if the engineer or architect should take those decisions instead, the problem is they won’t be able to answer for hours to days, during which no works would continue. That’s not feasible, so to keep things progressing you take your best guess, email your decisions to the architect and engineer, and if they dislike your choice enough hopefully they’ll let you know while it can still be fixed.
All this, plus the seventeen to eighteen hour days, is unsurprisingly very draining, though thankfully it’s been only for a maximum of four days per week so I’ve been able to recuperate in between these bouts of stress. This is the price of self building something atypical – the groundworks guys have been doing groundworks for over forty years, but they’ve never done ones like this before. They tell me they’ve greatly enjoyed themselves, rinse and repeat standard stuff is boring, and this job was definitely not boring, though they did find the pace frustratingly slow at times due to all the measuring and exactness. All this detail has also cost me dearly financially speaking, and I’ll tell you the total cumulative cost of all groundworks done towards the end of this diary entry.
Table of Contents
The Plan
Before recounting the story of these popups installation, the previous two rounds of groundworks were as follows:
September 2022: Installation of services: mains electricity, mains telecoms, mains water and mains foulwater along with small gravelled section for shipping container secure storage and site office.
April 2024: Installation of subsoil heat exchanger, removal of lots of soil, addition of dozens of tonnes of T2 structural gravel and permeable membrane. Due to all the soil removal and very expensive gravel added, this was an expensive round – though these past three weeks will probably cost about the same.
The original plan for this third round of groundworks (popups installation) was:
Total works this round:
- Nine foulwater popups.
- Seven inspection chambers.
- Four ESB ducts.
- Three stormwater popups.
- Three cavities.
- Two radon sumps.
- One mains water.
- Hundreds of meters of pipe/duct to connect between everything.
And this is it finished:
The Roof
The lads very kindly let me add the roof to their materials order and I went and visited all the building suppliers to get quotes to drive the price down to minimum, which was only possible thanks to it being a large order. Here are the three thousand or so tiles for the roof, with the fascia and soffit actually being behind the tiles next to the wall, but you can’t see that in this photo:
I reckon I saved myself about one third of the materials cost by doing this. Labour is by far the biggest cost in a build, so the effect on the overall cost is much less than one third, but it’s still a very nice saving. Unfortunately we ended up with a lot of hassle from this as the building supplier chosen was the cheapest for a reason – wrong items were supplied, items were missing, one of their trucks got stuck and I had to go find a local tractor to pull it out, the list went on in terms of inconvenience, hassle and stress.
But I did save hundreds of euro, so I guess my time and lack of sleep was worth it.
The Cave
Last round of groundworks we unfortunately hadn’t taken the depth of where the mains services enter the property, and therefore we couldn’t calculate the falls for any of the popups. So the very first item to do was to dig a hole at the front where the services enter, and get a measuring tape on them.
I was actually standing in that hole holding the mains water pipe out of the way of the digger’s arm when everybody panicked and told me to get out of the hole ASAP. It turns out I was quite literally standing over an underground cave, and the digger had just exposed an entrance exactly below where I had been standing:
The top of this underground cavity was about two metres down. It turned out to be four metres long, two meters wide, and about one metre high. It is almost exactly beneath the public footpath outside my driveway, and this is how it looks inside:
As much as it was surprising that we hadn’t discovered this during the last round of groundworks (we probably didn’t dig deep enough, and just missed it), underground cavities are known around this area: an underground river passes deep beneath this locality, and as it carries material away it undermines everything above it over time. We know that this footpath was built around twenty years ago, therefore around one metre of material is removed every twenty years. That meant that my driveway was likely to collapse before I would be dead, which meant I was going to have to spend a bunch of money I hadn’t intended. The solution was a reinforced concrete slab to bridge between the good bits of land so as the underground cave continues to deepen over time, my driveway should get held up until I’m dead (the public footpath and public driveway outside my property are my local government’s problem, not mine). Here you can see we backfilled part of the cave at the right, enough to pour concrete on top, then double layered steel mesh to form the slab:
We then filled the whole base with three cubic metres of concrete, which is oddly satisfying to watch getting poured:
We then dropped onto the reinforced concrete slab a heavy duty reinforced concrete manhole riser of the type they use for high traffic roads: this should stabilise the driveway above by taking weight from above and landing it evenly onto the concrete slab, so rather than weight pushing sideways it should push downwards:
Once that excitement was over, we now could calculate the levels for all the popups: anything taking a toilet flush was to have 1:80 fall, anything not taking a toilet flush was to have 1:40 fall. We’ll get back to that later, but here are both sewer lines being tested by my children pouring water into the inspection chambers and I in the big sewer making sure that the water flows:
Before anyone emails to mention that one side is unfinished and that will be a problem down the line, that side hasn’t been pointed yet. I may do that work myself as I need to mix a batch of cement anyway to patch up holes in the western wall before we paint it.
Land drains and cavities
We installed three land drains around the bottom of the edge of the T2 stone layer: one around the top of the tree cavity, one at the far left as once the house is up you won’t get a digger down there, and one around the bottom of the pool cavity:
Land drains are mainly there to prevent the build up of water which if it turned to ice, it would cause the T2 stone layer to expand, pushing up the house above. By digging a small ditch around the T2 stone layer, any water which gets into the T2 stone layer should flow into the ditch, and then the land drain spreads that around so it drains away. Similarly, if water tries to enter the T2 stone layer from outside horizontally, it will fall downwards and hit the land drain rather than enter the T2 stone layer. All this is great, however the land drain pipe is full of holes and not strong – it crushes easily. So you only ever install it last-most possible, which is exactly what we’ve done leaving the coils above ground for continuing installation later.
We dug out three cavities: the pool cavity, the pad cavity, and the tree cavity:
These are the pool and tree cavities: the pool cavity is the largest, but the tree cavity is the deepest:
The pad cavity is just 100 mm deep, but we had to spend enormous care here as most of the house services route around it, and we cannot disturb the subsoil underneath this as this pad takes the most weight of any pad in the house. So much weight in fact we need to triple mesh the concrete pad above it, and two large steel poles bear down onto this pad from above:
The sewer to the left is the master bedroom ensuite toilet, and the sewer to the right is the master bedroom ensuite shower plus upstairs bathroom. Right in between these two where the pad will go there will be his and hers sinks. When I’ll be brushing my teeth to go to bed, the most intense weight from above my head shall be load spread under my feet. Which is food for thought!
In case you are wondering why this pad takes the most weight anywhere in the house, it is because the rainwater harvesting tanks are above. When full, these add twenty metric tonnes to the rest of the weight above which includes the outer block leaf upstairs – those sit on metal girders which then puts as much load onto the ground floor outer block leaf as possible, but for the single corner where there is no other support that’s what the two steel poles and this pad support.
Popup installation over time
After each work day I had the drone take an aerial photo from thirty, forty and fifty metres up. I tried to find free of cost software to turn the thirty metre height photos into a timelapse video, but I couldn’t find any, so I had Step 3.7 Flash write a utility program which takes in photos, analyses them, chooses the best reference photo, and then scales-rotates-crops all the others to match. It also adjusts brightness so everything is roughly the same brightness over time, then emits a video in your choice of AV1, h.265 or h.264. Step 3.7 took about two hours to complete this task for a total cost of about US$0.66, and I only had to re-steer it once. The results are acceptable:
Here is the code it wrote: not bad methinks, I probably wouldn’t have done much better myself and furthermore I was and am not familiar with OpenCV for image analysis and processing, whereas it knew what it was doing relative to me:
#!/usr/bin/env python3
"""make_timelapse.py β Aligned AV1/H.264/H.265 timelapse from DJI aerial JPEGs."""
from __future__ import annotations
import argparse
import math
import os
import subprocess
import sys
from concurrent.futures import ThreadPoolExecutor
from typing import NamedTuple
import cv2
import numpy as np
# ββ Linearization (sRGB inverse EOTF) βββββββββββββββββββββββββββββββββββββ
def srgb_to_linear(val: np.ndarray) -> np.ndarray:
val = val.astype(np.float32) / 255.0
lo = val <= 0.04045
hi = ~lo
out = np.empty_like(val, dtype=np.float32)
out[lo] = val[lo] / 12.92
out[hi] = ((val[hi] + 0.055) / 1.055) ** 2.4
return out
def linear_to_srgb(val: np.ndarray) -> np.ndarray:
lo = val <= 0.0031308
hi = ~lo
out = np.empty_like(val, dtype=np.float32)
out[lo] = 12.92 * val[lo]
out[hi] = 1.055 * (val[hi] ** (1.0 / 2.4)) - 0.055
return np.clip(out * 255.0, 0, 255).astype(np.uint8)
# ββ Data structures ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
class FrameResult(NamedTuple):
filename: str
index: int # 0-based source index
M: np.ndarray | None # 2x3 similarity matrix (srcβref), None if skipped
inliers: int # RANSAC inlier count
total_matches: int
skipped: bool
skip_reason: str
warp_bgr: np.ndarray | None # uint8 BGR after warp
valid_rect: tuple[int, int, int, int] | None # (x0, y0, x1, y1) in ref space
crop_rect: tuple[int, int, int, int] | None # same
class TimelapseState:
def __init__(self) -> None:
self.last_output_bgr: np.ndarray | None = None
# ββ ORB matching (uint8 BGR grayscale) βββββββββββββββββββββββββββββββββββββ
def match_to_reference(
ref_gray: np.ndarray,
curr_gray: np.ndarray,
fringe_mask: np.ndarray | None,
) -> tuple[np.ndarray | None, int, int]:
"""Return (M, inliers, total_good_matches). M=None if insufficient matches."""
orb = cv2.ORB_create(nfeatures=2000, fastThreshold=5)
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=False)
kp_ref, des_ref = orb.detectAndCompute(ref_gray, None)
kp_curr, des_curr = orb.detectAndCompute(curr_gray, None)
if des_ref is None or des_curr is None or len(kp_ref) < 4 or len(kp_curr) < 4:
return None, 0, 0
raw_matches = bf.knnMatch(des_ref, des_curr, k=2)
good = []
for pair in raw_matches:
if len(pair) == 2:
m, n = pair
if m.distance < 0.75 * n.distance:
good.append(m)
if len(good) < 4:
return None, len(good), len(good)
src_pts = np.float32([kp_curr[m.trainIdx].pt for m in good]).reshape(-1, 1, 2)
dst_pts = np.float32([kp_ref[m.queryIdx].pt for m in good]).reshape(-1, 1, 2)
if fringe_mask is not None:
h, w = curr_gray.shape
y0, y1 = int(0.30 * h), int(0.70 * h)
x0, x1 = int(0.30 * w), int(0.70 * w)
def in_fringe(pts: np.ndarray) -> np.ndarray:
x = pts[:, 0, 0]
y = pts[:, 0, 1]
return ~((y0 <= y) & (y < y1) & (x0 <= x) & (x < x1))
fringe_src = in_fringe(src_pts)
fringe_dst = in_fringe(dst_pts)
keep = fringe_src & fringe_dst
src_pts = src_pts[keep]
dst_pts = dst_pts[keep]
if src_pts.shape[0] < 4:
return None, 0, len(good)
M, inlier_mask = cv2.estimateAffinePartial2D(
src_pts,
dst_pts,
method=cv2.RANSAC,
ransacReprojThreshold=1.0,
maxIters=2000,
confidence=0.99,
refineIters=10,
)
if M is None:
return None, src_pts.shape[0], len(good)
inliers = int(inlier_mask.sum()) if inlier_mask is not None else src_pts.shape[0]
return M, inliers, len(good)
def build_fringe_mask(h: int, w: int) -> np.ndarray | None:
y0, y1 = int(0.30 * h), int(0.70 * h)
x0, x1 = int(0.30 * w), int(0.70 * w)
edge_pixels = (
y0 * w
+ (h - y1) * w
+ y1 * x0
+ y1 * (w - x1)
)
if edge_pixels < 4000:
return None
mask = np.zeros((h, w), dtype=np.uint8)
mask[:y0, :] = 255
mask[y1:, :] = 255
mask[y0:y1, :x0] = 255
mask[y0:y1, x1:] = 255
return mask
# ββ Transform validation ββββββββββββββββββββββββββββββββββββββββββββββββββββ
def decompose_similarity(M: np.ndarray) -> tuple[float, float, float, float]:
tx = float(M[0, 2])
ty = float(M[1, 2])
s = float(np.sqrt(M[0, 0] ** 2 + M[1, 0] ** 2))
theta = float(np.degrees(np.arctan2(M[1, 0], M[0, 0])))
return s, theta, tx, ty
def validate_transform(M: np.ndarray) -> str | None:
s, theta, _, _ = decompose_similarity(M)
if not (0.90 <= s <= 1.10):
return f"scale={s:.4f} outside [0.90, 1.10]"
if abs(theta) > 5.0:
return f"rotation={theta:.2f}deg outside Β±5Β°"
return None
# ββ Pass 0: pairwise scoring ββββββββββββββββββββββββββββββββββββββββββββββββ
def score_pair(ref_gray: np.ndarray, curr_gray: np.ndarray, fringe_mask: np.ndarray | None) -> float:
M, inliers, total = match_to_reference(ref_gray, curr_gray, fringe_mask)
if M is None:
return float("inf")
cap_err = validate_transform(M)
if cap_err:
return float("inf")
if inliers < 4:
return float("inf")
return 1.0 / (inliers + 1)
def _score_row(
r: int,
n: int,
greys: list[np.ndarray | None],
fringe_masks: list[np.ndarray | None],
) -> float:
if greys[r] is None:
return float("inf")
total = 0.0
for j in range(n):
if j == r or greys[j] is None:
continue
total += score_pair(greys[r], greys[j], fringe_masks[j])
return total
def select_reference_frame(
files: list[str],
greys: list[np.ndarray | None],
fringe_masks: list[np.ndarray | None],
) -> int:
print("Pass 0: pairwise scoring to select reference frame...", file=sys.stderr)
n = len(files)
total_scores: list[float] = [0.0] * n
workers = min(os.cpu_count() or 1, n)
with ThreadPoolExecutor(max_workers=workers) as executor:
futures = {
executor.submit(_score_row, r, n, greys, fringe_masks): r
for r in range(n)
}
for future in futures:
r = futures[future]
total_scores[r] = future.result()
ref_idx = int(np.argmin(total_scores))
for i, sc in enumerate(total_scores):
print(f" {files[i]}: total_score={sc:.4f}", file=sys.stderr)
print(f"Selected reference frame: {files[ref_idx]} (index {ref_idx})", file=sys.stderr)
return ref_idx
# ββ Valid-content rect via mask warp βββββββββββββββββββββββββββββββββββββββ
def valid_content_rect(M: np.ndarray, h_src: int, w_src: int, h_ref: int, w_ref: int) -> tuple[int, int, int, int]:
mask_src = np.ones((h_src, w_src), dtype=np.uint8) * 255
warped_mask = cv2.warpAffine(
mask_src, M, (w_ref, h_ref), borderMode=cv2.BORDER_CONSTANT, borderValue=0
)
ys, xs = np.where(warped_mask > 0)
if len(xs) == 0:
return (0, 0, 0, 0)
return int(xs.min()), int(ys.min()), int(xs.max() + 1), int(ys.max() + 1)
# ββ Peripheral annulus mask ββββββββββββββββββββββββββββββββββββββββββββββββ
def peripheral_mask(h: int, w: int) -> np.ndarray:
y_top = int(math.ceil(0.10 * h))
y_bot = int(math.floor(0.90 * h))
x_left = int(math.ceil(0.10 * w))
x_right = int(math.floor(0.90 * w))
mask = np.zeros((h, w), dtype=bool)
mask[:y_top, :] = True
mask[y_bot:, :] = True
mask[y_top:y_bot, :x_left] = True
mask[y_top:y_bot, x_right:] = True
return mask
# ββ Brightness correction ββββββββββββββββββββββββββββββββββββββββββββββββββ
def compute_gains(
ref_linear: np.ndarray, frame_linear: np.ndarray, peri_mask: np.ndarray
) -> np.ndarray:
gains = np.zeros(3, dtype=np.float32)
for c in range(3):
r = ref_linear[:, :, c][peri_mask].astype(np.float32)
f = frame_linear[:, :, c][peri_mask].astype(np.float32)
denom = (f ** 2).sum()
if denom < 1e-12:
gains[c] = 1.0
else:
gains[c] = (r * f).sum() / denom
return gains
def apply_gains(frame: np.ndarray, gains: np.ndarray) -> np.ndarray:
return frame * gains.astype(np.float32)
# ββ Encoding βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
AV1_ARGS = [
"-c:v", "libsvtav1",
"-crf", "18",
"-preset", "5",
"-pix_fmt", "yuv420p",
"-color_primaries", "bt709",
"-color_trc", "bt709",
"-colorspace", "bt709",
"-movflags", "+faststart",
]
H264_ARGS = [
"-c:v", "libx264",
"-crf", "15",
"-pix_fmt", "yuv420p",
"-color_primaries", "bt709",
"-color_trc", "bt709",
"-colorspace", "bt709",
]
H265_ARGS = [
"-c:v", "libx265",
"-crf", "16",
"-pix_fmt", "yuv420p",
"-color_primaries", "bt709",
"-color_trc", "bt709",
"-colorspace", "bt709",
]
CODEC_MAP = {"av1": AV1_ARGS, "h264": H264_ARGS, "h265": H265_ARGS}
def check_ffmpeg() -> None:
try:
subprocess.run(
["ffmpeg", "-version"],
capture_output=True,
check=True,
)
except (FileNotFoundError, subprocess.CalledProcessError):
print(
"ERROR: ffmpeg not found. Install with: brew install ffmpeg",
file=sys.stderr,
)
sys.exit(1)
def check_av1_encoder(requested_av1: bool) -> str | None:
try:
result = subprocess.run(
["ffmpeg", "-encoders"],
capture_output=True,
text=True,
check=True,
)
except (FileNotFoundError, subprocess.CalledProcessError):
return None
encoders = result.stdout.lower()
if "libsvtav1" in encoders:
return "libsvtav1"
if requested_av1:
print(
"WARNING: libsvtav1 not found in ffmpeg build. "
"Attempting libaom-av1 fallback (slower).",
file=sys.stderr,
)
if "libaom-av1" in encoders:
return "libaom-av1"
return None
def build_ffmpeg_args(
width: int, height: int, fps: int, output: str, encoder: str | None, codec: str
) -> list[str]:
if encoder is None:
print(
"ERROR: No suitable AV1 encoder found in ffmpeg. "
"Install with: brew install ffmpeg (includes libsvtav1).",
file=sys.stderr,
)
sys.exit(1)
args = [
"ffmpeg",
"-y",
"-f", "rawvideo",
"-pix_fmt", "bgr24",
"-s", f"{width}x{height}",
"-r", str(fps),
"-i", "-",
]
if codec == "av1":
if encoder == "libsvtav1":
args.extend(AV1_ARGS)
else:
args.extend(["-c:v", encoder, "-crf", "30", "-cpu-used", "4", "-pix_fmt", "yuv420p",
"-color_primaries", "bt709", "-color_trc", "bt709", "-colorspace", "bt709"])
else:
args.extend(CODEC_MAP[codec])
args.append(output)
return args
def run_encoder(
frames_iter, width: int, height: int, fps: int, output: str, encoder: str | None, codec: str
) -> bool:
args = build_ffmpeg_args(width, height, fps, output, encoder, codec)
proc = subprocess.Popen(
args,
stdin=subprocess.PIPE,
stderr=subprocess.PIPE,
)
ok = True
try:
for bgr in frames_iter:
try:
proc.stdin.write(bgr.tobytes())
except BrokenPipeError:
stderr_text = proc.stderr.read().decode("utf-8", errors="replace")
print(f"ERROR: ffmpeg pipe broken. stderr:\n{stderr_text}", file=sys.stderr)
ok = False
break
finally:
if proc.stdin and not proc.stdin.closed:
proc.stdin.close()
stderr_text = b""
if proc.stderr:
stderr_text = proc.stderr.read()
ret = proc.wait()
if ret != 0:
print(
f"ERROR: ffmpeg exited with code {ret}.\n"
+ stderr_text.decode("utf-8", errors="replace"),
file=sys.stderr,
)
ok = False
return ok
# ββ Main pipeline βββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
def process_sequence(
input_dir: str,
output: str,
frames_per_image: int,
fps: int,
codec: str,
encoder_choice: str | None = None,
scale_factor: float | None = None,
) -> None:
check_ffmpeg()
requested_av1 = codec == "av1"
encoder = encoder_choice if encoder_choice else check_av1_encoder(requested_av1)
exts = {".jpg", ".jpeg", ".JPG", ".JPEG"}
files = sorted(
f for f in os.listdir(input_dir)
if os.path.splitext(f)[1] in exts
)
if not files:
print(f"ERROR: No JPEG files found in {input_dir}", file=sys.stderr)
sys.exit(1)
print(f"Found {len(files)} images.", file=sys.stderr)
# Load all frames (needed for Pass 0 pairwise scoring)
all_bgrs: list[np.ndarray | None] = []
all_grays: list[np.ndarray | None] = []
all_fringe: list[np.ndarray | None] = []
h_ref, w_ref = 0, 0
for fname in files:
path = os.path.join(input_dir, fname)
bgr = cv2.imread(path)
if bgr is None:
print(f"WARNING: Cannot load {fname}", file=sys.stderr)
all_bgrs.append(None)
all_grays.append(None)
all_fringe.append(None)
continue
h, w = bgr.shape[:2]
if h_ref == 0:
h_ref, w_ref = h, w
all_bgrs.append(bgr)
all_grays.append(cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY))
all_fringe.append(build_fringe_mask(h, w))
if h_ref == 0:
print("ERROR: Could not load any valid images.", file=sys.stderr)
sys.exit(1)
# ββ Pass 0: select reference frame βββββββββββββββββββββββββββββββββββββ
ref_idx = select_reference_frame(files, all_grays, all_fringe)
ref_bgr = all_bgrs[ref_idx]
ref_gray = all_grays[ref_idx]
assert ref_bgr is not None and ref_gray is not None
# ββ Pass 1: compute transforms to chosen reference ββββββββββββββββββββββ
results: list[FrameResult] = []
valid_rects: list[tuple[int, int, int, int]] = []
for idx, fname in enumerate(files):
src_bgr = all_bgrs[idx]
curr_gray = all_grays[idx]
if idx == ref_idx:
full_rect = (0, 0, w_ref, h_ref)
results.append(
FrameResult(
filename=fname,
index=idx,
M=np.eye(2, 3, dtype=np.float32),
inliers=0,
total_matches=0,
skipped=False,
skip_reason="",
warp_bgr=ref_bgr,
valid_rect=full_rect,
crop_rect=None,
)
)
valid_rects.append(full_rect)
continue
if src_bgr is None or curr_gray is None:
msg = f"corrupt JPEG"
print(f"WARNING: Skipping {fname}: {msg}", file=sys.stderr)
results.append(
FrameResult(
filename=fname,
index=idx,
M=None,
inliers=0,
total_matches=0,
skipped=True,
skip_reason=msg,
warp_bgr=None,
valid_rect=None,
crop_rect=None,
)
)
valid_rects.append((0, 0, 0, 0))
continue
h_src, w_src = src_bgr.shape[:2]
# Try fringe mask then full frame against chosen reference
fringe_mask = build_fringe_mask(h_src, w_src)
use_fringe = fringe_mask is not None
M, inliers, total = match_to_reference(ref_gray, curr_gray, fringe_mask)
if M is None and use_fringe:
M, inliers, total = match_to_reference(ref_gray, curr_gray, None)
skipped = False
skip_reason = ""
if M is None:
skipped = True
skip_reason = f"too few matches (got {total} good, 0 inliers)"
print(
f"WARNING: Skipping {fname}: {skip_reason}",
file=sys.stderr,
)
else:
cap_err = validate_transform(M)
if cap_err:
skipped = True
skip_reason = f"transform cap exceeded: {cap_err}"
print(
f"WARNING: Skipping {fname}: {skip_reason}",
file=sys.stderr,
)
else:
if inliers < 6:
skipped = True
skip_reason = f"only {inliers} inliers (minimum 6 required)"
print(
f"WARNING: Skipping {fname}: {skip_reason}",
file=sys.stderr,
)
elif inliers < 15:
print(
f"WARNING: {fname}: low inlier count {inliers} "
"(model may be unreliable)",
file=sys.stderr,
)
# Warn (do not reject) on large absolute translation vs reference
if not skipped and M is not None:
_, _, tx, ty = decompose_similarity(M)
if abs(tx) > 500.0 or abs(ty) > 500.0:
print(
f"WARNING: Frame {idx} ({fname}): large translation "
f"({tx:.1f}, {ty:.1f})px from reference β verify alignment",
file=sys.stderr,
)
if skipped:
results.append(
FrameResult(
filename=fname,
index=idx,
M=None,
inliers=inliers,
total_matches=total,
skipped=True,
skip_reason=skip_reason,
warp_bgr=None,
valid_rect=None,
crop_rect=None,
)
)
valid_rects.append((0, 0, 0, 0))
continue
# Warp to reference space
warped = cv2.warpAffine(
src_bgr, M, (w_ref, h_ref), borderMode=cv2.BORDER_REPLICATE
)
vcr = valid_content_rect(M, h_src, w_src, h_ref, w_ref)
results.append(
FrameResult(
filename=fname,
index=idx,
M=M,
inliers=inliers,
total_matches=total,
skipped=False,
skip_reason="",
warp_bgr=warped,
valid_rect=vcr,
crop_rect=None,
)
)
valid_rects.append(vcr)
# ββ Compute final crop rect βββββββββββββββββββββββββββββββββββββββββββββ
x0 = max(r[0] for r in valid_rects)
y0 = max(r[1] for r in valid_rects)
x1 = min(r[2] for r in valid_rects)
y1 = min(r[3] for r in valid_rects)
crop_w = x1 - x0
crop_h = y1 - y0
pad_w = (2 - crop_w % 2) % 2
pad_h = (2 - crop_h % 2) % 2
x1 += pad_w
y1 += pad_h
crop_w += pad_w
crop_h += pad_h
if crop_w <= 0 or crop_h <= 0:
print(
f"ERROR: Crop dimensions are zero or negative ({crop_w}x{crop_h}). "
f"Valid-content rects: {valid_rects}",
file=sys.stderr,
)
sys.exit(1)
ref_area = w_ref * h_ref
crop_area = crop_w * crop_h
if crop_area < 0.5 * ref_area:
print(
f"ERROR: Crop area ({crop_area} px = {crop_area/ref_area*100:.1f}% of reference) "
f"is below 50% guard. Aborting.",
file=sys.stderr,
)
sys.exit(1)
print(
f"Crop rect: x={x0}..{x1}, y={y0}..{y1}, size={crop_w}x{crop_h} "
f"({crop_area/ref_area*100:.1f}% of reference)",
file=sys.stderr,
)
for i, res in enumerate(results):
if res.valid_rect is not None and not res.skipped:
x0f, y0f, x1f, y1f = res.valid_rect
cc = (
max(x0f, x0),
max(y0f, y0),
min(x1f, x1),
min(y1f, y1),
)
old = results[i]
results[i] = old._replace(crop_rect=cc)
# ββ Pass 2: crop, brightness correct, generate output frames βββββββββββββ
state = TimelapseState()
peri = peripheral_mask(crop_h, crop_w)
out_w = crop_w if scale_factor is None else max(2, int(round(crop_w * scale_factor / 2)) * 2)
out_h = crop_h if scale_factor is None else max(2, int(round(crop_h * scale_factor / 2)) * 2)
ref_crop_bgr = results[ref_idx].warp_bgr[y0:y1, x0:x1]
ref_linear = srgb_to_linear(ref_crop_bgr.astype(np.float32))
def gen_output_frames():
last_output = ref_crop_bgr
for res in results:
if res.skipped or res.warp_bgr is None:
if state.last_output_bgr is None:
last_output = ref_crop_bgr
state.last_output_bgr = ref_crop_bgr
out = (
state.last_output_bgr
if scale_factor is None
else cv2.resize(
state.last_output_bgr, (out_w, out_h), interpolation=cv2.INTER_AREA
)
)
for _ in range(frames_per_image):
yield out
print(
f"WARNING: Frame {res.index} ({res.filename}): {res.skip_reason}. "
f"Substituting previous frame.",
file=sys.stderr,
)
continue
crop_bgr = res.warp_bgr[y0:y1, x0:x1]
if res.index == ref_idx:
corrected_bgr = crop_bgr
else:
frame_linear = srgb_to_linear(crop_bgr.astype(np.float32))
gains = compute_gains(ref_linear, frame_linear, peri)
extreme = ((gains > 1.5) | (gains < 0.5)).any()
if extreme:
print(
f"WARNING: Frame {res.index} ({res.filename}): "
f"extreme gains {gains.tolist()} β check exposure",
file=sys.stderr,
)
corrected_linear = apply_gains(frame_linear, gains)
corrected_bgr = linear_to_srgb(corrected_linear)
clipped = np.count_nonzero(
(corrected_bgr == 0) | (corrected_bgr == 255)
)
total_px = corrected_bgr.size
if total_px > 0 and clipped > 0.01 * total_px:
pct = clipped / total_px * 100
print(
f"WARNING: Frame {res.index} ({res.filename}): "
f"{pct:.2f}% pixels clipped after correction",
file=sys.stderr,
)
state.last_output_bgr = corrected_bgr
last_output = corrected_bgr
out = (
corrected_bgr
if scale_factor is None
else cv2.resize(
corrected_bgr, (out_w, out_h), interpolation=cv2.INTER_AREA
)
)
for _ in range(frames_per_image):
yield out
ok = run_encoder(
gen_output_frames(),
width=out_w,
height=out_h,
fps=fps,
output=output,
encoder=encoder,
codec=codec,
)
if ok:
print(f"Done: {output} ({len(files)} sources Γ {frames_per_image} frames each)", file=sys.stderr)
else:
print(f"ERROR: Encoding failed. Output may be incomplete: {output}", file=sys.stderr)
sys.exit(1)
def main() -> None:
parser = argparse.ArgumentParser(
description="Generate an aligned AV1/H.264/H.265 timelapse from DJI aerial JPEGs."
)
parser.add_argument("--input", default="./input", help="Input JPEG directory")
parser.add_argument("--output", default="./timelapse.mp4", help="Output video path")
parser.add_argument(
"--frames-per-image",
type=int,
default=30,
help="Output frames per source image (default: 30)",
)
parser.add_argument(
"--codec",
choices=["av1", "h264", "h265"],
default="av1",
help="Output video codec (default: av1)",
)
parser.add_argument(
"--fps",
type=int,
default=30,
help="Output frame rate (default: 30)",
)
parser.add_argument(
"--scale-factor",
type=float,
default=None,
help="Output resolution as a fraction of input dimensions (default: no scaling)",
)
args = parser.parse_args()
scale_factor = args.scale_factor
if not os.path.isdir(args.input):
print(f"ERROR: Input directory does not exist: {args.input}", file=sys.stderr)
sys.exit(1)
process_sequence(
input_dir=args.input,
output=args.output,
frames_per_image=args.frames_per_image,
fps=args.fps,
codec=args.codec,
scale_factor=scale_factor,
)
if __name__ == "__main__":
main()
Pressure testing the subsoil heat exchanger
The lads accidentally snapped the subsoil heat exchanger pipe twice and ‘bruised’ it a third time. So we needed to do a pressure test to make sure that the repairs weren’t leaking:
We plugged the mains water in into the subsoil heat exchanger, let the 150 metres of it fill with water, then let it wash out all the crap that was in there, finally we attached a radiator pressure tester. What this does is pump water in until you reach a desired pressure. You then leave it sit for a few hours and check that the pressure doesn’t drop, which would indicate a leak. We tested it at 100 psi, which is about seven bar. This is a bit much, to be honest, but the groundworks lads still work in imperial so 100 psi felt right to them. As all our fittings were plastic, they were right to use a higher test pressure, but the pipe itself is only rated to twelve bar or so. I personally would have used five bar, I think that enough for plastic fittings test (also it’s less hand pumping to get there and you get less pipe stretching so it’s way faster). Still, glad to know it’s all good at seven bar.
As built
This is my as-built overlay with the current site picture overlaid by the portion of items which were done in round three of groundworks. I personally measured the bottom of each foulwater popup and inspection chamber using my own laser level, plus the levels of each of the T2 stone layers throughout the site. I measured by sticking my tape measure down each popup and then taking where the laser appeared on the tape measure – note that for any popup less than 110 mm, I added 50 mm to the depth for any popup above a 90 degree bend, as my tape measure probably intersected the side of the bend and therefore read the depth too high. I have placed the intended plan picture from above directly below this one, so you can compare them easily.
Comparing these two, the leftmost inspection chambers are lower than planned, though the second down leftmost IC is relatively less lower, and therefore the fall between the topmost left two is half that planned. At the resulting 1:80 fall, we may get some soap and food residue dropping out between those two inspection chambers. The falls within the house to the IC are as planned, 290 mm was planned and 300 mm was achieved. The main bathroom toilet has a fall less than 1:80 which is unfortunate, however as it’s a 90 degree turn it should have higher velocity and I think it’ll be okay – I might stick my inspection camera down there, and see if water is pooling. The leftmost sewer line has a fall of 1:46, steeper than the planned 1:80, and then to the main sewer manhole it is also 1:46, where it should be 1:80.
For the rightmost sewer, the topmost IC is considerably lower than planned: a full 320 mm. The shower sewer has a steep fall to the ensuite toilet sewer of 1:23, then there is an even steeper fall to the IC outside of 1:19. This is obviously far steeper than 1:80. I didn’t lift the cover of the bottom rightmost IC as it’s covered with gravel, but I can say that the total fall from the top rightmost IC to the main sewer manhole is 1:68.
For the upper stormwater, everything is more or less spot on apart from the IC which being so deep was hand made from concrete and a two foot diameter concrete pipe. It’s a few dozen millimetres deeper than it ought to be, but not too bad.
Finally, the drone photo reveals that three popups were installed slightly out of place: the washing machine one is to the top right of where it should be, the small right ESB duct is to the bottom left of where it should be, and the bottom middle ESB duct emerges to the bottom left of where it should be. Of those, the only one which matters is the washing machine one, it’s now within the wall, but I’m sure we can build around that okay.
The soil pile excavated and materials added
As always, one is amazed at how much soil you dig out every time. Digging it out of course ‘fluffs’ the soil, making it take up twice the volume it used to be when in the ground. Still, that’s a fair soil pile, we basically were out of storage space:
There is at least ten truck loads of soil to remove, generally it is twenty metric tonnes per truck, so that is two hundred metric tonnes and that will cost me at least three grand ex VAT thanks to EU rules around soil transport.
Deposited into the ground was two truck loads of pipe grit, which is forty metric tonnes. A further three truck loads of T2 stone were spread around, which is sixty metric tonnes which is approximately 30 m3. So two hundred tonnes was removed and replaced with one hundred tonnes of expensive stone, leaving a net one hundred tonnes removed which are the three cavities shown above, and pipes installed.
In 2024 T2 stone cost β¬505 ex VAT per truck load. It now costs β¬600 ex VAT per truck load. I didn’t get a price for the pipe grit, it’ll be a bit cheaper, but certainly a good three grand went out on aggregate this time round. That’s way better than in 2024 where 18k of the then total bill was for material added and removed, but still a fair whack of money: between that and the soil removal, we’re talking six to seven thousand euro on trucks bringing or removing material.
Total cost
Back in the very early days we had β¬40k as the PC sum for site preparation and groundworks. Here’s what was actually spent:
- 2022: β¬6,810 inc VAT for two days of works.
- 2024: β¬25,537 inc VAT for five days of works, of which ~β¬18k was for aggregates and soil removal.
- 2026: Estimated β¬30,000 inc VAT for nine days of works, which includes maybe β¬5k worth of roofing materials and β¬7k for aggregates and soil removal, so maybe β¬18k went on labour plus materials and digger & dumper hire. The digger probably cost β¬550 per week, the dumper β¬300 per week, so maybe β¬2.5k went on hiring those and maybe β¬3k if including diesel. Another β¬5k might have gone on pipe materials etc, so β¬10k remains for wages, insurance, profit margin etc. Assuming a β¬400 per day worker cost including payroll taxes, that leaves about a β¬3k profit margin. That’s fair enough, I know I wouldn’t do a job for less than a ten percent profit margin.
So, if I remove the roofing materials, that’s a total cost of ~β¬57,000 for site preparation and groundworks, rather more than anticipated – though, to be fair, cost estimates pre-covid are barely comparable to today’s costs in anything, let alone construction materials.
What’s next?
We need a topographical survey of the as-built site, as my drone and laser level are only accurate within maybe 50 mm and a professional survey should come in within 10 mm.
We then need the builder to get some building onto this site! Tomorrow four years ago we got planning permission for this build. That means exactly ONE YEAR REMAINS until planning permission expires. So people really do need to get a move on now.
The second topmost left IC may be too high to allow toilet flushes to work correctly, if the engineer thinks so then that IC and surrounding pipes will need to be relaid. We’ll see what they say next week.
Finally, I expect next week my children will stain the left boundary wall with a ‘dark oak’ colour – I would call it ‘chocolate’ colour myself, but ‘dark oak’ is what it is called. It should take them four coats and therefore four days to complete: to save money, they’ll be applying three coats of the cheapest possible stain, followed by a fourth topcoat of the expensive stuff. The cheapest possible stain is not actually that terrible, it separates quickly at rest, so it needs a vigorous agitation before every time you use it. After that it is a bit thin, but that also means it dries quickly and it soaks into the wood well. Three coats onto bare wood produces an acceptable stain albeit still a bit blotchy (we know because we tested it on plywood sheets). The fourth coat of expensive stain should eliminate the blotchiness, also the expensive stain has some polyurethane in it so the final finish kinda has a plasticky varnishy feel to it, resembling a kind of matte varnish. I’ve used it before on park benches, you do get at least five years out of it and probably more – it begins to look raggedly after five years, but almost certainly would last at least another three years before peeling. So I believe their tin’s claims that you can expect ten years of wood protection. The cheap stain makes no such claims, it says three to five years depending on rain exposure. Fair enough for the price – it is less than one tenth the price of the expensive stain.
After they stain the left boundary wall, they have various camps for the next two weeks, then I intend to have them paint the right boundary wall in textured masonry paint. That stuff is a pain to work with, but it lasts forever on the wall and produces a very good finish. It is normally hideously expensive, but Screwfix will sell you one tub per shop per two weeks at β¬50 inc VAT per tub as a loss leader. So, if you need five tubs as we shall need, you laboriously go accumulate them over months which I’ve been doing and I now have all the tubs I think we shall need to complete the job.
In between painting, especially if the weather is nice, I expect to repeat the e-bike jaunts we did last summer around North Cork. It’s such a pretty part of the world when in sunshine. Thankfully, we live right in the middle of it, and I intend to make the best of it for another summer.