Welcome to ned Productions (non-commercial personal website, for commercial company see ned Productions Limited). Please choose an item you are interested in on the left hand side, or continue down for Niall’s virtual diary.
Niall’s virtual diary:
Started all the way back in 1998 when there was no word ‘blog’ yet, hence ‘virtual diary’.
Original content has undergone multiple conversions Microsoft FrontPage => Microsoft Expression Web, legacy HTML tag soup => XHTML, XHTML => Markdown, and with a ‘various codepages’ => UTF-8 conversion for good measure. Some content, especially the older stuff, may not have entirely survived intact, especially in terms of broken links or images.
- A biography of me is here if you want to get a quick overview of who I am
- An archive of prior virtual diary entries are available here
- For a deep, meaningful moment, watch this dialogue (needs a video player), or for something which plays with your perception, check out this picture. Try moving your eyes around - are those circles rotating???
Latest entries: 
Word count: 18894. Estimated reading time: 89 minutes.
- Summary:
- A diary entry is provided. The release and performance of Deepseek v4 Flash is discussed. Comparisons are made with other LLMs through coding and summarisation tests. Predictions regarding AI evolution, future Apple hardware, and the impact of humanoid robots on employment are detailed. The diffusion of technology within Europe is also analysed.
Friday 7 August 2026: 00:01.
- Summary:
- A diary entry is provided. The release and performance of Deepseek v4 Flash is discussed. Comparisons are made with other LLMs through coding and summarisation tests. Predictions regarding AI evolution, future Apple hardware, and the impact of humanoid robots on employment are detailed. The diffusion of technology within Europe is also analysed.
Before we get into that though, yesterday my children finished painting the west wall white, thus concluding successfully the painting of around one hundred square metres of exterior wall. I think they did really great given their ages:
To complete a job like this over multiple days, it requires a focus and self control and willingness to see things through to when they are complete which I find lacking in most eighteen year olds, never mind much younger again. Well done Clara, Henry and Julia!
Cheap open weights AI leaps forward yet again!
You may remember that I was initially keen on Qwen3 Coder Next, it was rather slow on my ancient hardware but it did work. However I found myself thereafter mostly using Step 3.5 Flash rented from OpenRouter as it was surprisingly good at coding and agentic work, and it looks like I was early compared to most to realise this – however, then Step 3.7 Flash dropped, and it was better in every way however also twice as expensive for new input BUT now they had prompt caching implemented. Step 3.7 also emitted far fewer thinking tokens than Step 3.5, so all in all the actual cost paid dropped by about half, and I’ve found myself using Step 3.7 Flash for pretty much everything since its release as it had the best ‘bang for the buck’ from my testing i.e. Pareto optimum, and to be specific:
- It is not the most capable model by any means.
- It makes many mistakes in the code it writes.
- It can take multiple attempts to perform an edit or call a tool successfully.
- BUT if you apply multiple rounds of it checking its work it does catch 98% of the bugs and bad logic it writes and fixes them correctly.
- It is sufficiently cheap that I’ve spent a total of US$14 ever on it, and that’s despite it horsing through 160 million tokens …
- From my testing on my own actual use cases, it was the optimal cost-benefit choice of LLM for all tasks where the data it processed is public (I use a local small Gemma 4 LLM for anything processing data which isn’t already on the public internet)
- Things I really like about Step 3.7 Flash: it follows instructions well, it’s very hard to jail break it out of its system prompt, if you order it to be biased or non biased in its system prompt it does as it is told, and a 196b model is feasibly likely to be runnable on consumer hardware arriving soon, so it’s worth investing into mastering this class of LLMs as your daily driver.
Amazingly, it was only five months ago that Qwen3 Coder Next (Q3CN) landed; and just three months since Step 3.7 Flash landed. Now we have the final release of Deepseek v4 Flash, and here are those LLMs compared so you can see why everybody including me is so excited by this particular LLM release and why social media (or at least my view of it) has been jammed with Deepseek v4 Flash 0731 posts for the past week:
| Qwen3 Coder Next | Step 3.5 Flash | Step 3.7 Flash | Deepseek v4 Flash 0731 | Claude Fable 5 | ||
|---|---|---|---|---|---|---|
| Released: | Feb 2026 | Feb 2026 | May 2026 | August 2026 | June 2026 | |
| MoE weights (total-active): | 80b-a3b | 196b-a11b | 196b-a11b | 284b-a13b | 6t-a400b | |
| Max input context: | 262k | 262k | 262k | 1M | 1M | |
| Typical Openrouter input cost after prompt caching: | $0.103/M | $0.100/M | $0.053/M | $0.030/M | $3.36/M | |
| Artificial Analysis Intelligence Index: | 21.1 | 26 | 30.3 | 49.9 | 59.9 | |
| Artificial Analysis Analysis Index: | 36.2 | ? | 39.6 | 69.1 | 76.5 | |
| Artificial Analysis Agentic Index: | 8.8 | ? | 21.5 | 45.7 | 52.8 | |
| AA Omniscience Accuracy: | 15.8% | 23.9% | 25.4% | 37.2% | 61.4% | |
| AA Omniscience Non-Hallucination Rate: | 9.1% | 14.8% | 15.6% | 15.6% | 45.1% | |
| SciCode: | 32.3% | 40.4% | 40.0% | 49.9% | 60.2% |
For comparison, I placed in the final column the current best performing LLM anywhere which is Claude Fable 5. It is 112x times more expensive than Deepseek v4 Flash 0731! Until now the Pareto optimum Step 3.7 performed about half as well as state of the art – now you have something 80% as capable and for nearly half the cost of the previous Pareto optimum.
Deepseek v4 Flash 0731 is a model which at 284 billion parameters is still within the realm of near-future consumer hardware: by 2028, as you’ll see later on in this diary entry, your standard new Apple Macbook Pro from 2028 onwards is expected will include similar compute and memory bandwidth to a 2017-era nVidia Volta AI accelerator board. That should run a model like Deepseek v4 Flash well at around one hundred tokens generated per second, and maybe four thousand tokens parsed per second. That’s a good bit faster than my rented edition has been, so that’s more than fast enough for serious usage.
Between now and then, and especially as the price per token has just halved again with Deepseek v4 Flash 0731 while the capabilities took another leap forwards, it makes the most sense to rent. After this AI investment bubble bursts, I fully expect prices for renting LLMs to crash spectacularly, almost to the point of free … which may make buying local LLM capable hardware a tough ask especially if somebody invents an end-to-end cryptographically secure LLM execution engine, which I’m sure is just a matter of time. However, if your new Apple Macbook Pro just comes bundled in for no extra cost the capability to run local LLMs in the hundred of billion parameter class, then that’s a different matter again.
So, this new LLM aces the synthetic benchmarks. How does it perform in real life?
Test 1: Summarising a 19th century will
Replicating this test from last January, I had it summarise the raw transcribed will of my ancestor Walter Irvine which is early 19th century legal English full of transcription errors. Even the ancient llama 3.1 8b doesn’t do terribly at this, but at the time Claude Sonnet 4.5 did a great job. How do Deepseek v4 Flash 0731 and Step 3.7 Flash perform?
| Cloud full fat Claude Sonnet 4.5 400b? 8 bit? | Step 3.7 Flash 198b 8 bit | Deepseek v4 Flash 0731 284b 4 bit | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| (from the previous post, it was said at the time it was nearly instant) | Tokens per second ~220.4 tokens/s Token count 5987 tokens Cost $0.0091834875 Duration 27.2s | Tokens per second ~178.6 tokens/s Token count 4347 tokens Cost $0.00182941902 Duration 24.3s | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
This is a comprehensive probate document from Edinburgh dated November 17, 1825, concerning the estate of Walter Irvine, Esquire, who died on January 7, 1824, at Luddington House, Egham, Surrey. He held properties in the Island of Tobago (West Indies), Luddington House in Surrey, and Denino in Fife, Scotland. Scottish Estate Inventory (Total: £5,258 6s 2d)I. Rents and Arrears Due (£4,569 13s 2½d):
II. Bond from John M. Harlaw of Kirkton: £245 5s 1d plus interest to January 1824 (Total: £262 15s 1½d) III. Bond by Road Trustees (Anstruther-Crail to St. Andrews): £150 principal plus interest (Total: £162 7s 4½d) IV. Bond by Lundenfoot Road Trustees: £50 with compounded interest from 1821-1824 (Total: £55 1s 2d) V. Proceeds from sale of wood: £33 10s VI. Balance due from Walter Dickson WS (Writer to the Signet): £174 16s 11d plus interest (Total: £174 18s 11d) Will Provisions (dated October 18, 1823)Primary Beneficiaries:Wife - Catherine Irvine (née Gordon):
Three Legitimate Daughters (each receiving £35,000):
Natural Daughter - Betsey (of London):
Other Significant Bequests:Ann Stewart (sister-in-law and widow of Stewart):
Brother's Estate:
Nephews:
Nieces (Clara and Marian Irvine - daughters of Christopher William):
Jane Armstrong (formerly Jane Currie, wife of Francis Armstrong):
Mary (reputed daughter of a negro woman named Linda in Tobago):
Other Named Beneficiaries:
Continuing Annuities:
Trust Structure and Powers:Executors and Trustees:
Powers Granted to Trustees:
Specific Provisions for Children's Trusts:
West Indies Property:John Leith Dispute:
Plantation Management:
Scottish Property Requirements:Election Doctrine:
Residuary Estate:After all debts, legacies, and annuities paid, the residue divided equally among three legitimate daughters as tenants in common:
Elizabeth's share for her separate use with power to dispose by deed or will despite coverture. Probate Details:Proved: January 28, 1824, in Prerogative Court of Canterbury Initial Grant of Administration to:
Reserved Power for: John Hamilton, Christopher Irvine, and Thomas Bruce to apply later Witnesses to Will:
Codicil: Dated same day (October 18, 1823), adding £100 bequest to Susannah Ann Hamilton Scottish Confirmation: Presented November 17, 1825, before George Carshin Jr., Commissioner of Edinburgh, with deposition by W.R.K. Douglas confirming the inventory and will details. The document demonstrates extensive wealth with significant properties in Scotland, England, and the West Indies, careful provision for family members including an illegitimate daughter, and sophisticated trust arrangements typical of wealthy estates in the early 19th century. |
Document Date: 17 November 1825 (Edinburgh) 1. Inventory of Personal Estate (Scotland)Presented by executor William Robert Keith Douglas (WS) and signed by him and George Carfrae Jr. (Commissioner). The total value of the Scottish estate is £5,258 / 6 / 2 (£5,258.31).
The testator also owned property in England; letters of administration for that part were already taken out by the executors. 2. Deposition (25 October 1825)Executor W. R. K. Douglas swore that:
3. Probate
4. Last Will and Testament – Detailed Summary4.1 General Directions
4.2 Provision for Wife – Catherine Irvine
4.3 Remission of Debt
4.4 Bequests to Individuals
4.5 Trust for Natural Daughter Betsy
4.6 Main Trusts for Daughters Christina Charles and Catherine Grace
4.7 Investment and Management Powers of Trustees
4.8 Devise of Real Estate
4.9 Executors and Guardians
4.10 Election Regarding Scottish Estates
4.11 Covenant for Daughter Elizabeth’s Marriage Settlement
4.12 Residuary Estate
4.13 Receipts and Discharges
4.14 Powers of Executors
4.15 Appointment of New Trustees
4.16 Indemnity of Trustees
4.17 Management Powers
4.18 Execution
5. Codicil (18 October 1823)
6. Affidavits and Probate DetailsHandwriting Affidavits (for the codicil)
Attesting Witness Affidavit
Grant of Probate
7. Key Figures at a Glance
This document is a comprehensive and intricate testament disposing of a substantial trans‑Atlantic estate, providing generously for the testator’s wife, daughters (both legitimate and natural), other relatives, and servants, while establishing elaborate trusts to preserve wealth across generations. |
This document is a collection of probate records concerning the estate of Walter Irvine, an Esquire who held property in the Island of Tobago (West Indies), Luddington House in Surrey, and Denino in Fife, Scotland. He died at Luddington House on 7 January 1824. The documents date from between late 1823 and late 1825 and comprise an inventory of his Scottish personal estate, a sworn deposition, his last will and testament (dated 18 October 1823) with a codicil, supporting affidavits, and the formal grant of probate from the Prerogative Court of Canterbury. I. Inventory of the Personal Estate (Scotland)The inventory was presented in Edinburgh on 17 November 1825 by Mr. Walter Dickson, Writer to the Signet, and made out by William Robert Keith Douglas, Esquire, one of the Lords of the Admiralty and an executor under the will (which was proved in the Prerogative Court of Canterbury in 1824, under the Act 48 Geo III cap 140). I. Rents and Arrears of Rent due to the deceased
II. Balance of debt due by John Hay Harlaw, Esquire of Kirkton
III. Sum in bond by the Trustees of the Anstruther & Crail to St. Andrews road
IV. Sum lent to the Trustees of the Lauderfoot road
V. Proceeds of roupes of wood due at death: £33. 10. 0VI. Balance due by Walter Dickson WS on account current (31 Dec 1823)
Total value of the deceased's estate in Scotland: £5258. 6. 2
II. Deposition (Edinburgh, 25 October 1825)Before George Carfrae Junior, Commissioner appointed by the Commissaries of Edinburgh, William Robert Keith Douglas swore that:
III. The Last Will and Testament (18 October 1823)The will revokes all prior wills, codicils, and testamentary dispositions. It directs all just debts, funeral expenses, probate costs, and legacies to be paid promptly out of the personal estate, with the real estate charged for any deficiency. Provision for the Wife — Catherine Irvine
Remission of Debt
Bequest to Ann Stewart (sister-in-law)
Other Bequests
Trust for Natural Daughter "Betsy"
Bequest to Jane Armstrong (formerly Jane Currie)
Bequests to Nieces Clara and Marian Irvine
Bequest to Mary (reputed daughter of enslaved woman Linda)
Annuities to Others
Bequests to Nephews
Main Legacies for Daughters Christina and Catherine
Powers of Trustees (Maintenance and Advancement)
Investment Powers
Devise of Real Estate
Appointment of Executors and Guardians
Election Requirement for Scottish Estates
Covenant Regarding Daughter Elizabeth's Marriage Settlement
Residuary Estate Distribution
Receipts, Powers of Executors, New Trustees, Indemnity, Management
Execution and AttestationExecuted on 18 October 1823 at Luddington House, signed and sealed by Walter Irvine across 19 sheets, in the presence of witnesses:
IV. CodicilDated 18 October 1823, adds a bequest of £100 to Susanna Ann Hamilton, daughter of John Hamilton, Esquire. V. Affidavits and Grant of ProbateAffidavit (Codicil)
Affidavit (William Adams, subscribing witness)
Grant of Probate (28 January 1824)
VI. CertificateLondon, 12 November 1825 — William Robert Keith Douglas certifies that this is the probate of the will of the late Walter Irvine referred to in his 25 October 1825 deposition on the inventory. Key Points at a Glance
|
Scroll down to see more ...
The good news is that it was much harder for me to determine the strengths and weaknesses of each of the two models compared to Claude from last year: Claude Sonnet 4.5 definitely still wins by a long shot, but the gap is much smaller than it was for Qwen 3.0 30b. Something which piqued my interest is why the Chinese models are so keen on outputting blow-by-blow structure of the original document, and I wondered if it is an artefact of the Mixture of Experts (MoE) design. So I also tested Gemma 4 31b which is dense and Gemma 4 28b-a4b which is MoE, and indeed the same blow-by-blow structure appears for the latter. I guess that kinda makes sense? Incidentally, Gemma 4 31b did surprisingly poorly on this test, I had assumed it would beat Deepseek v4 Flash as the Gemma models are well known to be better at English language nuance than the Chinese models, and while yes it did very well at picking out the right essential points from the will, it didn’t pick enough of those essential points despite being told to be detailed. Maybe I needed to say ‘very detailed’? Don’t get me wrong, the quality of Gemma 4 31b’s output was good, but it was short and to the point as it were, and too much short and to the point in fact.
Re: our two models, I think Step 3.7 produces a better structured documents – it is keen on tables – and it is more terse than Deepseek v4 which gives too much irrelevant detail, plus it writes English better in my opinion: less fluff, more densely packed. Deepseek v4 on the other hand did cost one fifth the amount which Step 3.7 did, and it’s not that much worse. Still, Step 3.7 wins this test on quality of output, if you exclude Claude.
Claude from last year is much better at synthesising the document together e.g. it groups daughters together, it has realised one is illegitimate, it orders items in a reasonable priority for most human readers, and it has collapsed all the multiple sections from the original into the minimum possible set. The Chinese models, despite getting towards a similar 400 billion parameters of Claude from last year, have a way to go yet, assuming that they’ll ever get there as they have a MoE design.
Test 2: Analyse a code implementation of a specification and implementation plan
Last few weeks I have been working on atomic_wait() for C, which essentially
ports the same feature from C++ 20 into the next C standard – though we shall
be adding some additional APIs, as we don’t care much for the C++ API. Myself
and fellow committee member Jens Gustedt came up with a draft WG14
proposal paper over a number of weeks, then I iterated having Step 3.7 Flash write
a detailed implementation plan for a reference implementation using another
hand written reference implementation for a separate WG14 proposal as a template.
It did struggle a bit with writing out the plan, and I had to hand hold it a fair
bit, but we got there.
The single most important part of the plan file is probably this which describes when a proxy atomic must be used which is indexed via an internal hash table, or whether the atomic wait can be passed through to the platform specific API directly:
| Backend | 1 byte | 2 bytes | 4 bytes | 8 bytes | Hash table needed? |
|---|---|---|---|---|---|
Linux (FUTEX_WAIT/FUTEX_WAKE) |
✗ | ✗ | ✓ | ✗ | For 1-2-byte and 8-byte; futex is 32-bit only (int *uaddr, int val) |
macOS (UL_COMPARE_AND_WAIT/UL_COMPARE_AND_WAIT64) |
✗ | ✗ | ✓ | ✓ | For 1-2-byte, or sub-native-width types |
Windows (WaitOnAddress) |
✓ | ✓ | ✓ | ✓ | Never — all operand sizes bypass |
FreeBSD (UMTX_OP_WAIT/UMTX_OP_WAKE) |
✗ | ✗ | ✓ | ✓ | For 1-2-byte, or sub-native-width types; UMTX_OP_WAKE accepts a count parameter directly |
pthreads fallback (pthread_cond_wait) |
✗ | ✗ | ✗ | ✗ | Always — no kernel tracker exists |
✓ = kernel primitive available; hash table is bypassed. ✗ = no suitable kernel primitive; must use the user-space hash table.
So, the design’s essential points are:
- There are multiple implementation backends for each platform specific API.
- The public API is able to pass through to the kernel API directly for some or all atomic types depending on backend.
- For the remaining types, an internal hash table maps an atomic’s address in memory to its proxy atomic which IS compatible with the kernel API.
I asked Step 3.7 Flash to implement the reference library using the plan and proposal as guides. It replicated over the mildly changed parts of the hand written source template well enough, but really struggled with designing and writing the implementation. It came up with a poor architecture and lots of repeated and verbose code all of which was unnecessary. So I ripped out all of that, and hand wrote an implementation for Mac OS which has compile time replaceable backends for different platforms. I then asked Step 3.7 Flash to implement the backends for (i) Microsoft Windows (ii) Linux (iii) FreeBSD and (iv) generic pthreads using the Mac OS backend as a template.
Here it did better, though it messed up the calling protocol, swapped a few arguments around, got error handling wrong etc. All not great, but then Deepseek v4 Flash 0731 was announced, and I thought it might be more interesting to see how both models would analyse the current implementation. I asked both this prompt:
Perform a comprehensive review of the implementation, comparing it to the plan and proposal. Write out any deviations, problems, bugs or any other issue into docs/deepseek_v4_flash_0731_analysis.md. Be detailed, also be aware that there is likely a race condition in the implementation which is causing lost wakes. Try to identify the cause without debugging.
Step 3.7 Flash took about four minutes to complete its analysis, taking
fifteen steps and costing US$0.075 dollars to process 101k tokens and
output 23k tokens. Deepseek v4 Flash unfortunately took my ‘try’ literally,
and launched gdb as part of its analysis. So I refined the prompt just
for it:
Perform a comprehensive review of the implementation, comparing it to the plan and proposal. Write out any deviations, problems, bugs or any other issue into docs/deepseek_v4_flash_0731_analysis.md. Be detailed, also be aware that there is likely a race condition in the implementation which is causing lost wakes. Identify the cause without debugging i.e. do NOT run gdb, do NOT run test programs - inspect and analyse the code ONLY.
… which worked, and it took ten minutes to complete its analysis, taking sixteen steeps and costing US$0.029 dollars to process 81k tokens and output 39k tokens. That’s about 2.6x cheaper, incidentally.
Its analysis was deeper, but it missed some easy issues Step 3.7 caught:
| Step 3.7 Flash 198b 8 bit | Deepseek v4 Flash 0731 284b 4 bit | |||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Step 3.7 Implementation Analysis1. Deviations from Plan/Proposal1.1 Windows 1-byte and 2-byte operations use hash table fallbackPlan: "Windows: always bypass (all sizes 1–8 bytes). The Windows backend never uses the hash table." Actual: Files: 1.2 macOS
|
| Issue | Severity | Location |
|---|---|---|
Hash table proxy stuck at atomic=1 after notify |
Critical | atomic_wait_common.ipp.ipp:220-222 |
| Lost-wake race: notify before park in hash table path | Critical | atomic_wait_common.ipp.ipp:41-45 |
atomic_wait_generic returns -1 instead of looping on proxy error |
High | atomic_wait_common.ipp.ipp:411-418 |
| Lost-wake race: notify without store in native path | High | atomic_wait_linux.c.ipp:43-44 |
Windows notify_all uses WakeByAddressSingle |
Medium | atomic_wait_windows.c.ipp:100 |
| Windows 1/2-byte ops use hash table instead of bypass | Medium | atomic_wait_common.ipp.ipp:524-528 |
wait_on_address32 discards errno |
Medium | atomic_wait_linux.c.ipp:50-51 |
macOS notify_all ignores max_threads_to_wake limit |
Medium | atomic_wait_macos.c.ipp:99-106 |
| No tests for hash table path | High | test/atomic_wait_test.c |
| No tests for notify-without-store | High | test/atomic_notify_test.c |
The most severe issue is the hash table proxy state machine (section 2.1 / race 3.1). The proxy is a single-slot binary semaphore that is set to "notified" (1) by the notifier but never reset to "waiting" (0) after consumption. This makes the hash table path fundamentally broken for any object that receives more than one notify, or for any waiter that arrives after a notify has already been issued for that object.
Deepseek v4 Flash (0731) — Implementation Review
Date: 2026-08-01
Scope: Reviewed the wg14_atomic_waits implementation against
docs/proposal.md (the authoritative spec) and plans/wg14_atomic_waits-reference-library.md.
Method: Static inspection of every include/ backend, src/ wrapper, the public
header, config.h, the tests, the build files, and CI. No debugger was run and no
test program was executed; all conclusions come from reading the code.
1. Executive summary
The implementation is structurally faithful to the proposal (all six public APIs exist, width dispatch for 1/2/4/8-byte atomics, native wait/notify backends with a hash-table fallback), but it contains one primary correctness race that produces lost wakes (Section 2), a broken pthreads backend (Section 3), a timeout smuggled in as an error on Linux (Section 4), a Windows lost-wake for multi-thread wakeups (Section 5), plus a number of smaller deviations from the plan/proposal.
The single most important finding: the per-object "proxy" used by the hash-table path sets a 0/1 notification flag that is never cleared while the wait-queue node is alive. This is the lost-wake/livelock race the task asked to identify.
2. PRIMARY RACE: the proxy notification flag is never reset → lost wakes / busy-spin
Location
include/wg14_atomic_waits/detail/impl/atomic_wait_common.ipp.ipp:
..._WAITmacro (lines 37–38):wait_on_address32(&(x)->atomic, 0, (timeout)) /* wait while proxy->atomic == 0 */..._WAKEmacro (lines 39–45):atomic_store_explicit(&(x)->atomic, 1, release), /* mono-directional: 0 -> 1 only */ wake_by_address32(&(x)->atomic, max_threads_to_wake)atomic_wait_generic()(lines 327–435) — the shared "park by proxy" loop.atomic_notify_generic()(lines 437–456) — the shared "set flag + wake" path.
The bug
A waiter parks by calling WAIT(item,...) which blocks while item->atomic == 0
(i.e. FUTEX_WAIT(&item->atomic, 0)). A notifier sets item->atomic = 1 and wakes.
There is no code anywhere that ever writes item->atomic back to 0 while the
wait-queue node is alive. The only place it is reset is at node creation inside
hash_table_find_or_create() (lines 220–222), which happens only when a brand-new
proxy_waiter_t is allocated. A node is freed only when use_count drops to zero.
Consequence — two interleaving outcomes
(a) Re-park after a wake never sleeps (livelock). Once any notify has fired on a
node, item->atomic is stuck at 1 for as long as the node lives. Any waiter that is
woken and must re-park — the proposal explicitly requires re-park on spurious wake, and
the code implements it as the top of the loop — calls
FUTEX_WAIT(&item->atomic, 0) while the value is already 1. The kernel compares
1 != 0 and returns EAGAIN immediately. Every subsequent iteration of the loop
returns immediately, so the waiter never sleeps again; it degenerates into a tight
100%-CPU busy-spin for the whole remaining lifetime of that node.
(b) Wakeups are lost because there is no sleeping thread. Because (a) means waiters
stop sleeping, a later genuine producer store + notify_* sets the flag (already 1)
and issues FUTEX_WAKE, which has nothing asleep to wake. The notify is therefore
effectively lost for the purpose of the sleep/wake contract; correctness then depends
entirely on the busy-spin poll observing the value change, which is not the semantics the
proposal defines and not what a correct reference implementation should do.
Why the analogous futex idiom would be safe but this one is not
The correct pattern guards the "am I allowed to sleep" decision on the same state that the notifier flips, and the notifier re-arms the state before waiting:
- waiter:
s = counter.load(); if (value == expected) futex_wait(&counter, s); - notifier:
counter++; futex_wake(...)Herecounteris a strictly increasing generation so the waiter can always detect a change that happened between its load and its sleep. The implementation instead uses a single 0/1 flag that is never re-armed, so the invariant "atomic == 0⇔ a notify is pending/expected" is destroyed after the first notify.
Which configurations suffer
This path is the fallback for every backend whenever the operand cannot be handled directly by the kernel primitive, i.e. exactly the cases the proposal/plan force through the hash table:
- Linux: 1-, 2- and 8-byte operands (
HAVE_WAIT_ON_ADDRESS_*is 32-bit only). - macOS / FreeBSD: sub-native widths (1/2-byte).
- pthreads backend: every operand (there is no kernel per-address waiter).
The 4-byte Linux/macOS/FreeBSD/Windows fast paths and the atomic_wait_expected
native-width path bypass the proxy and are not affected by this flag, but the
8-byte-on-Linux case — a perfectly legal and likely test target — is affected.
Recommended fix direction
Replace the 0/1 flag with a monotonically increasing sequence number that the waiter reads before parking and passes as the futex compare value, and that the notifier increments before waking. Reset-on-rearm must happen on the waiter side before the sleep decision, under the same lock used to re-check the object value (or rely on the kernel re-check for the object value itself as the futex fast path already does).
3. pthreads backend is fundamentally broken (hangs / lost wake)
include/wg14_atomic_waits/detail/impl/atomic_wait_pthreads.c.ipp:
..._WAIT(lines 45–46) ispthread_cond_wait(&(x)->atomic, pthreads_mutex()).pthreads_mutex()(lines 61–73) returns a_Thread_localmutex, i.e. a different mutex object per thread.
Problems:
pthread_cond_waitrequires the passed mutex to be held by the calling thread. Inatomic_wait_genericthe waiter has released the hash-table lock (line 387) and then enterspthread_cond_waitwith a mutex that is never locked. This is undefined behavior; on glibc it typically fails immediately (EPERM) so the wait “succeeds” without ever blocking — again a busy-loop — and there is no guarantee the node is protected.- The broadcast hand-off is not protected by the mutex the waiter sleeps on. A
notifier holds the global hash-table lock and calls
pthread_cond_signal(via the..._WAKEmacro, lines 47–53). The classic lost wake occurs when the notifier signals between the waiter’s re-check (value still equal toexpected, line 380) and itspthread_cond_wait: the signal is dropped and the waiter blocks forever. With a futex, the kernel’s value re-check/EAGAIN saves this; withpthread_cond_tthere is no such guard and there is no predicate/flag protecting the check, so the wait is a genuine, permanent lost wake (a hang). - Even the
INIT/DESTROYmacros treatpthread_cond_tthrough the genericproxy_waiter_t.atomicslot, but the sharedatomic_wait_genericstill performs flag-style logic (settinguse_count, etc.) that is meaningless for a condvar.
Because CI runs ALWAYS_USE_PTHREADS_BACKEND=ON on Linux and macOS, this path is
exercised, but its crashes/hangs are exactly the class of lost-wake bug being reported.
4. atomic_wait_expected mis-reports a timeout as an error on Linux
include/wg14_atomic_waits/detail/impl/atomic_wait_common.ipp.ipp,
atomic_wait_expected_32() (lines 595–666), plus the Linux
wait_on_address32() (atomic_wait_linux.c.ipp lines 37–52).
- The Linux
wait_on_address32returns0on success/EAGAIN/EINTR and-1on any other error, not-errno. A genuine time-out ofFUTEX_WAITtherefore comes back as-1(witherrno == ETIMEDOUT). - The caller’s error branch (lines 651–659):
if(ret2 < 0) { if(duration != NULL && ret2 != ETIME && ret2 != ETIMEDOUT) { errno = -ret2; return -1; } }ret2is-1, which is never equal to the positiveETIME/ETIMEDOUTconstants, so the condition is always true for anyret2 < 0when a duration was supplied. A clean time-out returns-1(error), not0(time-out) as the proposal requires:Returns: … returns zero … or duration timeout occurs.
This is timing-dependent — if the pre-wait clock_gettime check (lines 631–637) happens
to notice expiry first it returns 0 cleanly — so the 1 ns test in
atomic_wait_expected_test.c is flaky, but the underlying error path is wrong.
5. Windows wake_by_address* only ever wakes a single thread → lost wake
include/wg14_atomic_waits/detail/impl/atomic_wait_windows.c.ipp,
wake_by_address32/wake_by_address64 (lines 92–124):
if(WakeByAddressSingle((PVOID)(uintptr_t) object)) return (max_threads_to_wake == 1) ? 1 : 1;
return 0;
- The
max_threads_to_wakeparameter is ignored; bothatomic_notify_allandatomic_notify(..., max_threads_to_wake=N>1, ...)call this and wake exactly one thread viaWakeByAddressSingle. The correct routine formax != 1isWakeByAddressAll. Every other waiting thread is left parked → lost wake. - The
(max_threads_to_wake == 1) ? 1 : 1ternary is dead code (both branches are1).
This makes the Windows backend incorrect for atomic_notify_all and for
atomic_notify with max_threads_to_wake > 1, which the plan marks as mandatory
behaviour.
6. macOS timeout conversion deviates from the plan
atomic_wait_macos.c.ipp, wait_on_address32/64 (lines 55–68, 77–90):
- The plan (Step 11) requires:
*duration→ nanoseconds, cap eachulock_waitcall atUINT32_MAX(~4.29 s) and loop for longer durations. - The implementation instead converts once to microseconds
(
tv_sec * 1000000U + tv_nsec / 1000U) and passes it in a single call with no cap and no loop. For any duration ≥ ~4295 s the microsecond value overflowsuint32_t, and durations beyond ~4.29 s are not split across multiple calls, so the accumulated wait can be far shorter than*duration— violating the proposal’s “total accumulated time … shall be at least*duration”. - The code also declares private
extern __ulock_wait/__ulock_wakeinstead of including<bsd/sys/ulock.h>as the plan directs; functional risk if SDK/version behaviour differs.
7. FreeBSD 8-byte UMTX_OP_WAIT argument-order inconsistency
atomic_wait_freebsd.c.ipp:
- 4-byte:
_umtx_op(object, UMTX_OP_WAIT_UINT, expected, (long)&umtx_time)(lines 57–58) — passes the expected value in the value slot. - 8-byte:
_umtx_op(object, UMTX_OP_WAIT, (long)&umtx_time, (long)expected)(lines 88–89) — passes the timeout pointer in the value slot and the expected value in the address slot, i.e. the two are swapped relative to the 4-byte call.
The two calls are internally inconsistent, so at least one passes the operands in the
wrong order; UMTX_OP_WAIT (8-byte) is almost certainly wrong and will compare against
garbage / misbehave.
8. Return-value deviations from the plan
atomic_notify_32(lines 668–691) returns1 + reton a successful CAS, whereretis the number actually woken (0 on the proxy path when no node exists, 0 on the futex path when nothing is parked). So CAS-success-with-no-waiters returns1(positive), whereas the plan explicitly states:atomic_notify… Returns 0 if the CAS fails or no waiters are parked.(The proposal’s “possibly one plus the number woken” makes
1defensible, so this is a plan deviation, not a proposal violation — noted for completeness.)atomic_wait_genericsetsret = 1(line 388) before the first actual park call. The proposal ties “positive” to “suspended at least once”; the intent-to-park flag is acceptable but slightly loose.
9. Width-dispatch macros silently no-op on unsupported widths
atomic_wait.h, _WG14_ATOMIC_WAITS_IMPL_atomic_wait* / _notify* (lines 107–201):
- Each macro is
if (sizeof==1) … else if (==2) … else if (==4) … else if (==8) …with no finalelse(anddo{}while(0)). A_Atomictype of any other width (e.g. 16 bytes, or along double, or a 0-width type) compiles to a silent no-op rather than a compile-time error, which can mask misuse. The plan states widths 1/2/4/8 are supported; a diagnostic would be safer.
10. Header-only / ODR notes
- The
atomic_wait_*_N/atomic_notify*_Ndefinitions in the.ippfiles are not themselves markedWG14_ATOMIC_WAITS_INLINE/static(only their prior declarations inatomic_wait.hcarryWG14_ATOMIC_WAITS_EXTERN, which isinlineonly whenWG14_ATOMIC_WAITS_ENABLE_HEADER_ONLY). In the multi-TU header-only build (header_only_test*.cpp) this relies on the C inlineextern-inlinemodel. It is workable on GCC/Clang but fragile; the plan required every backend helper to be explicitlystatic inlineto guarantee ODR safety. hash_table()usesWG14_ATOMIC_WAITS_IGNORE_MULTIPLE_DEFINITIONS(weak), so the singleton table is fine across TUs; this part is OK.
11. Smaller issues
errnonot preserved on success/failure paths ofatomic_wait_expected_32: it writeserrno = ETIMEDOUT(line 636) anderrno = -ret2(line 656), clobbering the caller’serrnoeven thougherrnopreservation was a stated plan requirement (Steps 8–12). The futexwait_on_address32does restoreerrno, but the caller overwrites it afterwards.EINTRtreated as success (return 0) inwait_on_address32(Linux line 45) rather than retrying; a signal during a timed wait silently shortens it.atomic_notify_genericdrops a notify when no node is registered (find_or_create withincrement_use_count=false, lines 444–445). For the futex backends this is masked by the waiter’s under-lock re-check; for the pthreads backend it is not masked (see Section 3), which is where the drop becomes a hard lost wake.- Hash-table growth/deletion re-probes correctly (cluster-shift deletion, lines 271–
280), but
hash_functruncates a 64-bit pointer to 32-bit before mixing (lines 108–113); not a correctness bug, just avoidable collisions.
12. Structure deviations from the plan (non-bug)
- The plan (Step 7) specified that
atomic_wait/atomic_wait_explicitbe thin wrappers delegating toatomic_wait_expected, andnotify_one/notify_alldelegate to awake_waitershelper. The implementation instead uses per-widthatomic_wait_1/2/4/8+ direct kernel calls for the native widths, with the hash-table path only for the fallback. This is a reasonable and arguably more correct choice given thatatomic_wait_expectedis native-width-only, but it is a deviation from the plan’s described structure. benchmark_atomic_wait_test.cdoes not actually run a benchmark (returns 0 after declaring an unused consumer), so CI excludes it correctly but it validates nothing.
13. Conclusion
The implementation gets the public API shape, the width dispatch, and the native-width
futex path right, but the hash-table/fallback proxy logic has a one-way notification
flag that is never re-armed, which is the race responsible for lost wakes: waiters that
re-park on a node that has already been notified stop sleeping and busy-spin, and
subsequent notifications have no sleeping thread to wake. On the pthreads backend the same
logic is additionally broken by a thread-local mutex and a missing predicate under
pthread_cond_wait, producing hard lost-wake hangs. These are compounded by a Linux
timeout→error bug, a Windows single-wake bug for notify_all/max>1, a macOS timeout
conversion that violates the minimum-duration guarantee, and a likely FreeBSD 8-byte
argument swap. All were identified by code inspection only.
Scroll down to see more ...
Differences:
- Only Step 3.7 noticed that the Windows backend doesn’t bypass for all atomic sizes as it should according to the implementation plan.
- Step 3.7 false positives ‘atomic_notify_generic creates a proxy even when no waiters exist’.
- Step 3.7 side mentions the lack of a generational count: Deepseek v4 gets properly into the issue and describes it in full and does it well at that.
- Deepseek v4 has a much better handle on what’s broken in the pthreads backend and why.
- Only Deepseek v4 spots the inverted arguments in the FreeBSD backend.
Also, in general, the Deepseek v4 report reads more coherently and gets into the big issues quickly and tersely, whereas the Step 3.7 report is bitty and kinda all over the place.
Neither did a good job of identifying where the implementation plan or the implementation deviate from the proposal. The implementation plan specifically states at its top:
docs/proposal.mdis the authoritative spec. Behavior, return values, and memory-order semantics must match it exactly.
After this I stopped using Step 3.7 and exclusively used Deepseek v4. Perhaps the latter was overwhelmed by all the defects and deviations from specification with this first analysis which is why it didn’t perform well – all I can say is that later on, perhaps as the implementation’s major bugs got fixed which made logic analysis easier, Deepseek v4 Flash began to seriously impress me with its analysis capabilities. One still has to go through multiple rounds of something like:
Exhaustively compare the implementation to the proposal, looking for all cases of deviation, bug, issue, concerns and corner case where the implementation does not match the proposal, or the proposal may not be implemented in full. Be very detailed, thorough and methodical in your approach - go that extra mile each and every time. Analyse in depth all implementation backends and all combinations of build configuration options, also analyse in depth all error handling and control flow paths not normally taken. Update plans/combined-analysis with your findings.
… and then you rinse and repeat iterations of that, fixing one by one all the things it finds, and doing so until it finds nothing important in your opinion. So in that sense it’s like Step 3.7, but where it massively improves is on one shot fixes for CI failures: you simply tell it which CI failed and copy and paste the failure text. It then had a 100% success rate at finding and fixing the CI failure even for platforms it could not debug locally – it did so simply by inspection and analysis, including inspecting online the kernel sources for Linux, FreeBSD or ReactOS (to get an insight into Windows).
In fact, there was an especially impressive bit where it found that
Apple Clang 17.0 only would produce invalid binaries if symbol visibility
was set to hidden and a specific tail call optimisation caused a
function to be inlined into main(). It went off decompiled the Apple
Clang binary, compared it against the LLVM clang source code, found
the exact bit of problematic reverse compiled source code, wondered to
itself it it ought to patch the Apple Clang binary, and eventually decided to
instead hack around the problem for this specific instance and it even
added an informative explanatory comment to say why its hack was there.
Now, I’d read of Claude doing stuff like that. I’d seen Step 3.7 analyse
the assembler in binaries to figure out why they weren’t performing
as expected. But to actually localise a bug in a third party precompiled binary via reverse engineering?
That was new to me. No doubt it did take rather a long time to do all
that – due to Deepseek v4’s immense popularity right now, it has not
been running quickly, as little as 30 toks/sec. But I could leave it
chug away on its own safely I found, whereas Step 3.7 had a nasty
habit of occasionally wrecking your git repo or going off and installing
huge bits of software it didn’t really need via brew.
After the WG14 atomic waits reference library was finished, I had spent US$5.02 on 465 million tokens. That is US$0.0108 per million tokens. Yes that is an awful lot of tokens – in fact, I have consumed 913 million tokens ever on OpenRouter, so this one project consumed half my lifetime total – but the quality of implementation created is very high in my opinion. I would estimate it would have taken me over one hundred hours to create a similar quality implementation by hand – instead this cost me less than ten hours in total, which was almost entirely spent reviewing its work and giving direction on what to do next. Five dollars for ninety hours of my life back to do more interesting work is a bargain.
Test 3: Subjective experience of using each LLM to get work done
I think it’s fair to say I’ve been repeatedly wowed by Deepseek v4 Flash 0731’s capabilities. I HAVE found that you should not let it take architecture direction decisions: always ask it to present a menu of implementation options, and you’ll find half the time its recommended implementation is the wrong one. So that part sucks. But when you choose on its behalf the right implementation option, 98% of the time it does a great job: it matches the style and form of the existing codebase, it avoids writing copy and paste code and instead hoists common routines into reasonable locations in reasonable common header files, and the code quality written is well above most of the programmers I’ve ever worked with, with only very occasional slip ups. I really like the much improved one-shot fix capability, especially for platforms and architectures I can’t run on my system where I have the LLM agentic harness running. That’s been a HUGE timesaver: no more having to boot up Windows VMs etc to diagnose some random failure on CI.
I very much like its performance analysis. I asked it to make this codebase go faster. It spent some minutes pondering and reading code, and it told me we ought to use triangular probing instead of quadratic probing in the open addressed hash table as the buckets are a power of two, so the triangular probing would ensure better scattering of entries avoiding collisions. It one-shotted the new implementation, then benchmarked the difference, then twiddled a few unrelated items by parsing through the optimised disassembly as it knew my main ask was for improved performance. It then spat out hard benchmarks: 470 nanoseconds reduced to 50 nanoseconds. Impressive. It also generated comprehensive tests for scalability under load and that bucket growth did work perfectly under heavy multithreaded load. Even more impressive.
I asked it to add the Fil-C toolchain to the CI (this is a guaranteed memory safe C/C++ toolchain). It went off and found the documentation on the web, followed the instructions, set up the appropriate Github CI actions, adjusted the codebase where necessary as the Fil-C libc is musl rather than glibc, then to test it it installed a Linux VM as this is a Macbook, installed the correct AArch64 edition of Fil-C rather than the x64 one the CI uses, and ran the test suite via the local Linux VM under Fil-C. Worked first time in a single shot too – it didn’t make a single mistake. Had I done that by hand, I definitely would have made a mistake at least once – I know from past experience that setting up the Fil-C toolchain is finicky.
I originally had asked Step 3.7 to create this new reference library using an existing hand written reference library as its template. It didn’t do too well at that, and with hindsight I wish I’d have wiped what it did and started from scratch with Deepseek v4. Deepseek v4 Flash makes far fewer mistakes within the test harness (Kilo Code) and doesn’t need to self correct anything like as much. It also gets tool calling in the Kilo code harness right almost all of the time, unlike Step 3.7. I suspect it would have done a better job at mimicking the template into this new library, but I guess I’ll not find out until the next time I write a new reference library for WG14.
Deepseek v4 Flash is much more prone to proactively fix bugs and issues
without being explicitly told it can first. It seems to ask for forgiveness
rather than permission. So you’ll need to be careful to always git commit
before asking it a question, otherwise it may decide your question demands
code changes. At least it doesn’t like to git reset --hard
as Step 3.7 Flash was keen on doing when it got the codebase into
a confused state, rather it properly uses git stash so any working
tree changes can be recovered.
The 1M max context of Deepseek v4 Flash makes a BIG difference! I was used to beginning to sweat as the 260k context limit approached, trying to get it to write out todo lists into Markdown files for the next session clear as I always found context compaction just didn’t work well with Step 3.7. With Deepseek, I can just relax and let it trundle on – you are still wise to start a new session from time to time as the long contexts slow its execution down, but now you can take your time about it, and more importantly, if it goes off on a long extended think or ponder or diagnosis of something you can just ignore it because it won’t suddenly run out of context. This is the first occasion I can go do some other task while it runs in the background and I don’t need to stress constantly checking its progress. Very nice!
And finally, I really like how much cheaper it is. On Openrouter comparing usage and spend from before to after, I issued twice the requests and spent half as much money. I didn’t have to babysit this model as much as before. All in all, this is my new favourite Pareto cost-benefit optimum LLM choice. Well, at least for the next three months, if the pattern so far this year continues to hold true!
Which brings me onto …
Where LLMs and AI probably are going next
We now have enough history of LLM evolution to be able to predict with reasonable reliability where things will go next. As I mentioned above, I’ve chosen a new LLM coding assistant every three months on average this year. Shall I continue to do so?
I had Deepseek v4 Flash go off and scrape the AA Intelligence Index for a spread of LLMs over the past two years off https://artificialanalysis.ai/, then plot those using a contour map:
This has contour bands for ventiles in a LLMs AA Intelligence Index score and it shows that:
- For a <= 10 billion parameter model which is feasible for me to run locally given my ancient hardware, we broke into the tens around January this year, and we would expect to break into the twenties any time around now, followed by the thirties around January 2026, and the forties around Summer 2027. So, by Summer 2027, a 10 billion parameter model will score as well as Deepseek v4 Flash does. And it’ll run well even on this old Apple M3 based laptop.
- For a <= 300 billion parameter model which is likely to run well on near future Apple Macbook Pros, we broke into the twenties around last November, into the thirties last March, into the forties last week, and we would expect to break into the fifties before the end of 2026, then into the sixties by March 2027. Reminder: Claude Fable 5 scores sixty-one. So, before the end of Spring 2027, that six trillion parameter model will score similarly to a 300 billion parameter model!
Let’s graph time directly against AA Intelligence Index score:
What strikes you about this graph is firstly by how much the open weights models are catching up with the closed weights ones – I would be surprised therefore if the Chinese government continue to release their frontier models with downloadable weights in the near future. Secondly, there is a clear structural break between 100-200b and 200-500b models – there is wide space between their trend lines. Indeed, right now 200-500b is outperforming 500-1T, which is surprising. Less surprising is the 1T+ category which has a trend line matching that of the closed weight models.
The current most intelligent LLM anywhere by this index as of August 2026 is Claude Opus 5 with a score of sixty-one, followed by the current most intelligent free to download LLM which is Kimi K3 with a score of fifty-seven (Kimi K3 is a cool 1.4 Tb of download, and generally you need about as much VRAM as the download size to run it, so that would be very expensive to run locally given current RAM prices). Deepseek v4 Flash 0731 is 167 Gb to download, so it would run well on a machine with 256 Gb of VRAM, and it gets an intelligence score of 49.9. Of course, a single score is an average, and some models are strong and weak on specific domains compared to others. https://artificialanalysis.ai/ lets you compose comparisons of LLMs, so I chose these six as representative for this discussion:
Scroll down to see more ...
As you can see, the AA Intelligence Index score is made up of lots of separate indices, each of which is then weighted into an average overall score. On some specific domains e.g. GPQA Diamond or r3-Banking, already it’s a wash between recent models. On some others, there is a clear pattern of the gap rapidly narrowing, however there are always going to be things at which a five trillion parameter model will beat the pants off a 500 billion parameter model: tool use and logic aren’t those, but specialist knowledge and reasoning will be.
In other words, yes while the AA Intelligence Index score will improve over time for smaller models, that will be in those parts of the index which aren’t specialist knowledge and reasoning. Small models simply can’t store as much knowledge as larger ones, so I expect my estimations above to be rather optimistic i.e. the score improvements will be less for the smaller parameter models than one would currently predict by extrapolation from recent past. Especially because the AA Intelligence Index score is out of one hundred, so as models max out all parts except the specialist knowledge and reasoning, they will end up running into an upper bound where only more parameters can improve some of their domain specific scores.
ALSO all this prediction is contingent on the AI investment bubble continuing to inflate. One gets cleverer small models by investing more compute into training fewer parameters. For that, you need more and cheaper compute, and for that you need to keep investing those billions. This year they think ~US$900 billion has been invested in AI, and for next year we are on track for US$1.4 trillion dollars in 2027. Even the great wealth and income of the tech multinationals will struggle to fund so much debt – even now, total free cash flow for several of them is below their debt servicing costs for the debt they’ve taken out. But that’s another diary entry. In any case, it is hard to believe that the AI investment bubble won’t pop soon, and then we’ll have whatever compute has been built out by then and it’ll only grow linearly rather than exponentially after that, much like with the late 1990s telecommunications infrastructure investment bubble.
Linear compute growth does still enable model improvements, and unsurprisingly I’d expect them to stop improving exponentially and start improving linearly instead. As with the end of Moore’s law, the slowdown will affect the biggest highest end models first, and the smaller lower end models will see a long run of continuing exponential growth before that eventually also peters out. It’s been the same with CPUs: at the very cheap end, they’ve been continuing to exponentially improve the value per dollar cost for decades after the high end went into linear improvements. I think the same will apply to LLMs: after all, if training cost per dollar goes from exponential to linear improvement, the lessons learned from making the high end a little better should translate into larger improvements at lower ends, same as for CPUs.
I find this prediction of the future FAR more believable than predictions of imminent Technological Singularity which have started doing the rounds again. I covered that in the unpublished book I wrote after St. Andrews: the Singularity is purely the result of an artefact of human perception where we tend to weigh more recent big leaps forward as more important, as they are more important to us personally but aren’t really in the bigger picture of things outside humanity. Elon Musk had an interview with the Economist week before last where he was banging on about the Singularity. I suppose that suits his purposes to market that philosophy aggressively so fewer think about seizing some of his trillion dollars of personal wealth, but I also got the impression from the interview that he actually genuinely believes that a Singularity will happen at some point. I’ll categorically state right now: no Technological Singularity will happen in my lifetime unless some very new technology turns up. Certainly nothing about Large Language Models as presently designed and implemented is capable of generalised artificial intelligence i.e. AGI. Right now we’re in the exponential growth phase because we’re pouring exponential amounts of capital in – cut the constantly increasing capital investment flows and you can say good bye to exponential LLM capability improvements, as I just described above. All that said, the near term improvements to consumer hardware WILL be significant to Economic Total Factor Productivity as the gains from this technological advancement begin to diffuse widely throughout society.
Near future hardware
So that brings me onto the near future consumer hardware to run these things. Recent leaks say that Apple have started to design their next M-series and A-series chipsets to have better than the usual trendline of improvements to compute and memory bandwidth, so your 2028 Apple Macbook Pro should locally run < 500 bn parameter LLMs quite well indeed. Here are the current rumours and leaks in a single table and graph as created from the table by Deepseek v4 (at which it was surprisingly poor at doing interestingly, I really had to poke it hard and repeatedly to generate correct looking SVG, despite it amazing performance at graph building shown above – maybe the HTML table input upset it?):
| Model / Year | Edition | GPU Cores | Memory Bandwidth | Remarks |
|---|---|---|---|---|
| Apple M3 2023 | Pro | 18 cores | 154 Gb/sec | Unfortunately my personal Macbook is the M3 Pro, the worst for running LLMs of any of the Pro Macbooks 🙁 |
| Max | 40 cores | 410 Gb/sec | ||
| Apple M4 2024 | Pro | 20 cores | 273 Gb/sec | Added a memory cache shared between CPUs and GPUs like AMD's Infinity Cache for its GPUs. This greatly improved latency. |
| Max | 40 cores | 546 Gb/sec | ||
| Apple M5 2026 | Pro | 20 cores | 307 Gb/sec | First with hardware matrix multiply and accumulate (= nVidia 'tensor cores'). LLM input parsing is approx 4x faster than M4 as a result. 1024 FP16 FMAs per core per cycle enables 70 FP16 TFLOPs for the Max edition. |
| Max | 40 cores | 614 Gb/sec | ||
| Apple M6 2027? | Pro | 32? cores | 512? Gb/sec | Expected move to LPDDR6 standard memory architecture featuring a wider 24-bit channel layout (shifting to 384-bit Pro / 768-bit Max buses) to achieve a projected 1.67x generational leap in raw memory speeds. There will be no Max nor Ultra edition of the M6, this suggests that the core will be very similar to the M5 and they only upgrade the memory bandwidth. |
| Max | 64? cores | 1024? Gb/sec | ||
| Apple M7 2028? | Pro | 48? cores | 800? Gb/sec | Rumours say the Max variant can be fitted with up to 768 Gb of RAM in your standard Macbook laptop chassis. Obviously so much RAM will be VERY expensive as Apple likes to charge steeply for additional RAM. It would be surprising if TFLOPs don't double due to implementing 2048 FP16 FMAs per core per cycle. |
| Max | 96? cores | 1600? Gb/sec |
If you extrapolate out the numbers, the Apple M7 Pro should have the same memory bandwidth as a nVidia Volta enterprise AI accelerator from 2017, and the M7 Max should have the same memory bandwidth as a nVidia Ampere AI accelerator from year 2020. Chances are that the Macbook Pro and especially Max will have more VRAM (or equivalent, see below), but in terms of compute with 1024 FP16 FMAs per core per cycle they should pretty much match a Volta and Ampere exactly: the Volta maxed out at 125 FP16 TFLOPs and the Ampere 312 TFLOPs. Both had hardware matrix multiple and accumulate, same as the Apple M-series from the M5 onwards. As mentioned in the table above, it would be surprising if the M7 doesn’t implement at least 2048 FP16 FMAs per core per cycle given that today’s nVidia Rubin chipset can do 16384 FP16 FMAs per core per cycle, and Apple tends to follow closely whatever architecture choices nVidia makes – the M5 chipset’s GPU looks awfully like a nVidia GPU, just less wide. This architectural closeness is why LLM software support tends to be nVidia first, then Apple, then AMD (which is architecturally different), then Intel (which is architecturally different again). And why LLM software support on Apple is first class, whereas although support for AMD has improved enormously, it remains second class.
There are zero rumours about this next bit, so it’s probably wrong, but I would wonder if Apple would fit so much DRAM when flash mounted as Storage Class Memory (SCM) is (i) cheaper and especially (ii) much less drain on battery life. There is zero good reason why LLMs are stored in DRAM other than there isn’t an easily available cheaper substitute, but somebody big like Apple could simply fit NAND flash where the DRAM goes. You might only write that flash with an updated LLM every few months so its endurance won’t matter, and NAND flash if mounted like RAM is nearly as fast as DRAM. As the LLM model weights aren’t mutated in RAM, this could save easily 80% of the RAM demands of a LLM, so you get to run your 200 Gb sized LLM in 40 Gb of DRAM and probably less if you shrink the size of the KV cache which is very doable if you have Ampere levels of compute on tap.
Obviously that’s pure speculation, but I do know that DRAM is hard on battery life as it must be continually refreshed. Storage class memory would be easy to fit for somebody big like Apple and it would fix the battery life impact problem. I guess we’ll find out in 2028. In any case, you would expect parsing of around four thousand tokens per second, and generation of a hundred tokens per second on the M7 Pro – and double that for the M7 Max. That’s very acceptable for an ultrabook sized laptop.
Diffusion of local LLM capable hardware throughout society
Lots of ink both physically and virtually has been spilled lamenting how Europe isn’t keeping up with the US and China on AI advancement: we aren’t investing in the electricity supply for datacentres, nor in AI research past a small fraction of what the Americans and especially the Chinese are doing. It is therefore claimed that Europe will be left behind, and left at a significant disadvantage to the US and China.
This kind of claim has been made many times before on many topics of industrial, social and political comparison between the three superpowers – and it is true that especially recently Europe has felt on the back foot as it gets bullied simultaneously by the other two superpowers, which it isn’t used to historically. However, something less appreciated is that Europe is surprisingly good at diffusing more quickly and completely the gains of an advancement than the other two superpowers: it ‘buys in’ the advancement cheap, then mass disseminates it.
That will need explaining, so to simplify: Europe, due to its unique configuration of highly competitive constituent arms length states with huge size variations, tends to diffuse innovations faster and more broadly than America or China does. This is surprising on first inspection, but think of it this way: if Ireland obtains a large current account surplus by diffusing US sourced innovations widely across its economy, all cash strapped countries elsewhere in Europe start paying rapt attention and will try to duplicate and/or improve upon whatever Ireland is doing. Ireland gets a lot of stick internationally for being a tax haven and washing the profits of US multinationals of their tax obligations elsewhere – all of which is fair – but less appreciated is that all those US multinational operated subsidiaries in Ireland do genuinely diffuse US innovations throughout the Irish economy much quicker and more completely than they could in the US where they are nowhere near as relatively economically dominant. Same goes in Switzerland and Belgium incidentally.
Obviously I’m exaggerating a touch there – at times I do wonder about diffusion of best practices in Ireland – but my point is that in superpowers such as the US and China, practice of best practices tends to be concentrated in specific economic clusters such as New York or San Francisco-San Jose in the US, or Shenzhen-Guangzhou or Shanghai in China. Whereas Europe’s economic clusters are more geographically distributed and numerous in a unique three spoke configuration:
These are the famous blue, golden and green ‘bananas’ of European economic cluster (source). Unusually they all connect together through the North of Italy, which is exactly why while the Covid pandemic may have originated in China, it turned into a global pandemic in the North of Italy as that is the most connected place to other places in world bar none other, so all global pandemics will always spread worldwide from there. As with infectious diseases, so does the global diffusion and spread of new ideas and best practices all originate from Northern Italy.
And the same will undoubtedly apply to the mass adoption and use of LLMs: the US may design the hardware and the Chinese may manufacture the hardware, but it’ll be Europe who reaps the most economic value for the cheapest price from their inventions. This is why Europe always appears to be an economic laggard, yet by all metrics it has the best quality of life for the most people anywhere in the world despite having the lowest debt to GDP ratio of any of the world superpowers. Before some say ‘that’s because you don’t spend enough on defence’, I already debunked that in past posts here: Europe has rarely spent less than the US on a PPP adjusted basis, and last few years it is by far the biggest military spender in the world (and if you include Russia in Europe, which most would, then Europe has by far and away always spent more on its military than anywhere else in PPP terms). So, in terms of economic and welfare achievement, Europe’s practice of cheaply reaping from what others sow has served it very well.
How will this affect individual behaviours and mentality?
What will the world be like when your laptop and increasingly your phone locally runs a LLM as powerful or more powerful than the world’s currently most powerful LLM?
You might think what is different to the laptop or phone using a LLM running in a cloud elsewhere and using it over a data connection?, and in some ways you would be right: I’m using a Deepseek running in some cloud elsewhere over a network connection. What’s the difference between that and running it locally?
The first difference is privacy: I wouldn’t ever put anything potentially confidential anywhere near a public internet connection. I definitely wouldn’t put any personal emails or family photos near a public internet connection. Most people won’t care, so maybe this difference only matters to people like me. Still, I’m also an individual, and for me this matters a lot.
The second difference is cost centring: if a cloud runs the LLM, somebody has to pay for that and your average individual is highly adverse to subscriptions when a free of cost substitute is available. So 98% of individuals right now use the free LLM services, and they are generally terrible because otherwise they’d cost real money. If the LLM runs on your device, you take a hit to battery life, but otherwise it’s free of cost. So for your typical individual, from 2028 onwards they’re going to experience an enormous leap in LLM capability, as until then all they’ll be used to is the crappy cheap to operate free LLMs.
The third difference is that most businesses – and a fair few individuals – don’t like to introduce single points of failure to their operations. Most cloud services are seen by many as exceedingly annoying when they go down. And the more you depend on such a service, the more anxious you get if it could disappear/get cut off/drop out. If LLMs run exclusively on hardware you personally own and control, a lot of that anxiety lifts. Now you can lock yourself into this new technology with a certainty you couldn’t have had before. It is for this exact reason why private automobiles are so popular: you aren’t buying transport from A to B, rather you’re buying the guarantee of transport from A to B which public transport only offers in big cities.
The fourth difference is that if they’re truly free of cost and you can run them all day long and all it costs you is electricity, you’re going to use them a LOT more. For everything in fact. Why search the web if your local LLM can do it for you? Why order anything or reply to any message if your local LLM can do it for you? Why think about interacting with your device if your local LLM can do it for you?
And now we’re getting into the interesting stuff: what can a LLM automate away, and what can’t it do i.e. what role is left for humans?
We don’t still know how intelligent LLMs will become before the bubble pops, but I can say this: the LLM knows more about everything than you do, but not more about some specific topics than you do. Accepting on what topics you are weak but being honest about where you genuinely really do understand more than the LLM will be the key to your success going forth.
LLMs genuinely can be a force multiplier if you use them where their strengths lie, and combine that with your strengths. But they also hallucinate and are currently lousy at direction and strategy, so that’s where I would expect the value of humans to remain. In other words, I think politicians are going to have some of the best job security going forth, because their whole purpose is to set unpopular directions for everybody else.
How will this affect individual employment?
The future world of human employment I suspect is (a) those physical jobs which can’t economically be replaced by a robot controlled by a LLM and (b) those jobs where a human’s deep understanding of a niche topic of value cannot be surpassed by any LLM, or where decisions must be taken which involve long term direction and strategy. For everything else, I expect LLMs to gradually replace all before them.
Speaking of LLM controlled robots, I was quite surprised to discover that they only melded a LLM with a humanoid robot last June, so we’ve got a few years to go before humanoid robots start taking human physical jobs. But not as many years as you might think!
Much also to my surprise, it turns out that nobody was mass producing
non-toy humanoid robots until only November last year! Absolutely before
then as now you can buy toy humanoid robots,
these will dance for you
and do kung fu etc, but they’re absolutely useless for getting any real
work done as they (a) can’t lift enough weight reliably and safely and (b)
they don’t have the sensors for fine dexterity manipulation in unfamiliar situations.
And absolutely before now there were intelligent mass produced
industrial robots – any modern factory is stuffed with them – but none
were humanoid until last November.
The first mass produced industrial humanoid robot was the Ubtech Walker S2 which you can see to the right, and it went on sale in November 2025 and has probably sold about three thousand units. It costs about €150k ex VAT, it can carry up to 15 kg and you get about 2.5 hours per battery charge, though it can swap out its battery at a battery recharge station on its own so it can work continuously without a break. Its intended use is within pristine environments where fairly fixed programming works well e.g. walk over there, pick up one of X, rotate it until it has the right orientation, walk back here, put it into the right component box. In other words, just like any other industrial robot, but this one is capable of adapting to different locations within the same factory.
Next up is the Boston Dynamics Atlas which entered mass production in January 2026, and probably about two thousand units have been sold so far. It costs about €200k ex VAT, it can carry up to 30 kg including an impressive 20 kg if on one arm, and you get about two hours per battery charge. It seems a bit more intelligent than the Walker S2, but not by much: it is also intended for fixed, repetitive, work in a pristine environment like a factory floor. This robot is undoubtedly a lot more impressive in the build quality sense than the Walker S2, but it does cost a third more, and also Boston Dynamics only put it into mass production now after decades of development because they had to due to Chinese competition – not because it was finished or particularly compelling or priced well. It also is less interesting because all its production for the next two years is already sold to Hyundai, so nobody else will be able to buy one for several years more yet.
Last April, a much more interesting industrial humanoid robot went into mass production: the Figure 03. Here is a youtube live stream of it unpacking parcels in a mail office, ensuring that the address label points downwards for scanning:
Firstly, the handling of irregularly sized, sometimes squishy, items is FAR harder than the regularly sized boxes with grab handles that the previous two robots can handle. The Figure 03 can also climb stairs by itself – albeit slower than a very elderly person – but it does get there. It currently costs about €100k ex VAT, it can carry up to 20 kg and you get a very good five hours of battery life, but at the cost of it being 40% slower at movement e.g. it walks slower, moves slower etc. They have sold maybe four thousand of these by now. It comes with a bundled LLM running locally which isn’t particularly good – nowhere near even Deepseek v4 Flash in fluid conversation – but if you tell it to go wash the clothes in the washing basket it’ll go fetch the basket and take it to the washing machine, very slowly pick each item out and put it into the machine, then very slowly pour in detergent and set the washing machine running. Ultimately, apart from the thing getting in the way a lot due to its lethargy, it is a far more interesting humanoid robot – at least for the very wealthy, not least due to its cost, but also because you would really need a home with large open spaces so you can easily get around the robot while it very slowly does things. Just to be clear: the Figure 03 can run and jog as fast as a human, but it absolutely horses through its battery if it does, plus it gets hot – very hot! Still, maybe future firmware revisions could let it exchange battery for speed for short bursts so it isn’t annoying and doesn’t get in the way, but otherwise conserve battery life.
If I am being honest though, I suspect the Figure 04 is the one to wait for, as the Figure 03 feels like it has too many design and hardware compromises, and it is still too expensive for what you get on the software side. If it’s the most impressive in mass production right now, what screams out loudly is just how immature and unfinished its software story is. It’ll be years, at best, before that can be remedied.
In case you’re wondering what about all the other mass produced industrial humanoid robots, that’s it: everything else isn’t actually in mass production. In particular, Tesla’s very long advertised robot is nowhere to be seen: we don’t know its specs, its price, or anything else about it, and given Elon Musk’s long history of made up claims about autonomous driving, I wouldn’t be optimistic that his robot will have good autonomy for at least several years after launch. Ultimately this is because it’s one hard thing to build the hardware for an affordable price, it’s another hard thing to create compelling software for that hardware platform. As an example, Meta solved building affordable VR headset hardware, but they did not solve building a compelling software ecosystem for it, so the whole thing has gone off to die and it’s only a matter of time before that entire ecosystem is abandoned. Similarly, Tesla’s fully autonomous driving will likely never get solved well enough to be allowed by regulators at a price consumers will pay.
As much as Figure 03 is impressive, it still requires a pristine environment i.e. you can’t be taking it onto a building site. Even if a robot could navigate well such an irregular environment, and it coped well with getting mud and sand into its joints, it would almost certainly move too slowly for many tasks on a building site AND generally annoy the human construction workers by getting in the way.
Currently a construction worker might cost about €100k to the employer, so maybe for €100k a construction site robot might be worth the expense if it only did things like fetch concrete blocks so the blocklayers could keep working without pause. But as each block weighs 30 kg, it would need to be able to move wheelbarrows of them over scaffolding, which is far beyond the capabilities of any current or near future humanoid robot. You’d also need several of them as they’d go much slower than humans, and because they’d run out of batteries after a few hours you’d either need a quick battery swap facility or even more robots. And finally most construction sites don’t have electricity apart from a generator, so charging robots at a site would be very unattractive. So, for certainly the next decade, I think construction workers can rest in peace that they will not get made unemployed by humanoid robots.
For human jobs doing physical labour in pristine environments though, the next ten years looks like increasing levels of human jobs being displaced. If they can get the cost of these robots down to €25k, a lot of minimum wage jobs like stacking shelves or packing online orders look inevitably gone forever, as the minimum cost to the employer of a human (minimum wage is about €32k) makes the robot look cheaper. If somebody successfully cracks deep cleaning by robot, that’s all your cleaning staff gone too. Jobs like fast food kitchen work is at threat, even if the delivery driver is not – that’s a lot of your young person entry level jobs disappearing forever there.
The most recent (2024) ESRI report lists the largest number of minimum wage jobs being in these sectors:
- Kitchen helpers (14%)
- Shop sales assistants (10%)
- Bartenders (7%)
- Caretakers (6%)
- Waiters (6%)
- Home based personal care (3%)
- Housekeepers (2%)
- Receptionists (2%)
About ten percent of the Irish workforce earns near minimum wage, and humanoid robots could take over most if not all those eight sectors if they get cheap enough.
Food for thought indeed! This displacement of humans from their jobs by AI might have impacted IT first, but I am extremely sure it shall be coming for entire sectors of knowledge worker and pristine environment manual labour next.
Word count: 2666. Estimated reading time: 13 minutes.
- Summary:
- The as built survey is documented. A boundary wall was painted, and a water intrusion issue in the pool cavity was encountered during testing. Furthermore, reference levels were remounted, and the as-built survey data was compiled following the completion of groundworks.
Thursday 23 July 2026: 20:29.
- Summary:
- The as built survey is documented. A boundary wall was painted, and a water intrusion issue in the pool cavity was encountered during testing. Furthermore, reference levels were remounted, and the as-built survey data was compiled following the completion of groundworks.
That ended up being four coats of cheap stain and one coat of expensive stain: we probably should have applied a fifth coat of cheap stain to reduce the blotchiness by a bit more, but we were out of time as camps began the next day so I just had to draw a line under it. It is my no means bad, just not as evenly coloured as I would personally prefer. Clara and Henry did well, it was their first time painting anything big and given the hot weather, staying the course required will power. My son, aged nine, clearly was very done with painting by the end of the fifth day of it, but then he is aged nine. Both did good quality work, I only had minor touch ups to do and those were principally due to their lack of experience painting a wall. Total all in cost including paying them €150 between them was €300 inc VAT for twenty-four metres of wall, or €6.25 per sqm. That’s very good in my opinion for painting an unpainted wall which tends to soak up the paint first time.
The pool cavity becomes a pool!
An unpleasant surprise turned up part way through the left wall staining: the pool cavity became a pool! Here it is with the water already mostly pumped out:
The water I reckon was about 400 mm deep if you include the stone layer at the bottom, and about 330 mm deep from the top of stone within that pool cavity. Very obviously, the tree cavity – which is much deeper – was dry as a bone, and it has not rained to any extent in weeks. So it is a mystery how it filled up like that!
I took the opportunity to use the water to test my percolation test hole. While we were digging out the pool cavity, we found at the bottom some possibly wet soil whereas the tree cavity was dry as a bone all the way down. That got me curious, so I had the groundworks guys dig me as deep a hole as the digger would go right next to the pool cavity where the future soakaway was planned to get installed:
That’s about 1.6 metres deep, and you can see a little bit of ground water down there, so obviously the water table is somewhat high around here. My concern was that installing a soakaway two metres down would be pointless if it was below the water table in summer, and I can confirm that that little bit of water didn’t budge in the days after it was dug. When the pool cavity filled up, it still didn’t budge after several days, so I wondered what if I filled that percolation test hole and emptied the pool cavity. Here is the result after five days:
It may have dropped by an inch in five days during ~28 C constant sunshine weather, so I’m going to claim that the height loss was entirely evaporation. The pool cavity remained bone dry after it was pumped out, and as the top of water in the percolation hole is about half a metre above the bottom of the pool cavity, so clearly my subsoil has almost zero percolation. This was another unpleasant surprise, as I had specifically chosen this site expecting subsurface static water pressure to cause flow over my subsoil heat exchanger into the ditch just behind my property, making the heat exchanger more efficient. Apparently I have here the exact opposite 🙁.
My current working theory is that my neighbour may have doused his garden with water to keep it alive, and due to the lack of percolation all that water promptly poured through his upper layer of topsoil over his subsoil into the pool cavity. If this is the case – I shall be watching things carefully – then there is a danger that the RC box going into the pool cavity could float if enough water surrounds it. To calculate that:
Pool cavity including EPS: 5885 x 3730.
EPS thickness: 200, with density 40 kg/m3.
RC thickness: 200 at bottom 215 at sides, with density 2,500 kg/m3.
Empty middle: 5055 x 2900, with density 1.25 kg/m3 (air).
So:
- Total horizontal area = 21.95 m2.
- Horizontal area of RC box = 18.27 m2.
- Horizontal area of empty air within RC box = 14.66 m2.
- Horizontal area of RC box = 18.27 m2.
- Bottommost 200 mm of EPS = 4.39 m3 => 175.60 kg.
- Nextmost 200 mm of RC box = 3.65 m3 => 9,133 kg plus 30 kg for the outer EPS => 9,163 kg.
- Remaining H metres of RC box = H x (40 x (21.95 - 18.27) + 2500 x (18.27 - 14.66) + 1.25 x 14.66)
- H x (40 x 0.736 + 2500 x 3.61 + 1.25 x 14.66)
- H x (29.44 + 9025 + 18.33)
- => H x 9073 kg.
Therefore:
H x 2195 (water) = -(9339 + (H - 0.4) x 9073) (RC tank)
H = ~0.51 metres above the base of the RC box
As the pool cavity is 0.77 metres below the top of T2 stone, and the T2 stone is about 200-250 mm deep, any water entering the pool cavity should overflow under the rest of the house before the RC tank would float, so we’re safe.
That said: you better believe I’ll be running the land drain out of the pool cavity no higher than 0.5 metres from its bottom, just in case.
Remounted the reference levels
Many years ago now, I had screwed three pieces of wood to the wooden fence to mark 75.08 over sea level, and all works since have used that as the reference level. Unfortunately, one bit of wood split and so had to be removed, and another got put out of alignment due to the wood staining. That left me with just one reference level marker, which got me anxious due to the lack of redundancy, so before the professional surveyor arrived I remounted two additional reference levels matching the one remaining one.
As my laser level is an indoor only model, I had to wait to dusk to be able to see the laser:
I remember being very, very tired when doing this work as I had been up at 4am and having to stay up until dusk really sucked. But I knew I’d get a lie in the following day, and I think I ended up asleep that following day until about 1pm as I had built up such a sleep debt.
When the surveyor came, we found that the bottom of those reference markers is more like 75.07 rather than 75.08. Thankfully, it didn’t seem to affect the stone levels according to the same survey, they were all spot on or slightly lower. I was also rather glad to learn that my marker on the wall other side of the site was just 4 mm off: my laser level, a Bosch prosumer model, promised <= 0.02% height error per metre of distance – over 35 metres, you would expect the error to therefore be less than 7 mm, so it was well within claimed specifications and only 43% worse than a commercial laser level.
The as-built survey
My surveyor, knowing that I have been more than a year without income, very kindly offered to reduce his bill if I did the post-processing work of taking his raw data and adjusting it to calculate the locations of all the popups. To explain, the problem with a pipe sticking up is you can’t directly measure the X-Y location of its centre, rather you can measure its side and then you need to offset that X-Y location in to the centre of the popup by half its diameter plus 15 mm (which is half the width of the X-Y location measuring stick). My surveyor would normally do that work and charge me for his time, but he reckoned me technically able enough to do it myself. And I am most grateful to him for that, every euro saved counts right now. Here is my as-built image of the site overlaid on the most recent drone photo taken a few days ago:
The levels shown above for the pool cavity, tree cavity and steel pad cavity are all as built; each popup as built is shown as a small green dot with its deviation from plans in text next to it; and the foulwater (FW) and stormwater (SA) all show the bottom of each pipe or inspection chamber as built. As with the sewer levels being somewhat off in places by up to 80 mm, so were the X-Y popup locations where the deviation was between 20 and 70 mm apart from the ESB ducts which are 100 mm off. Those ESB ducts being so far off is annoying, but as they are ESB ducts you can give them repeated kicks to get them back into position without disturbing the falls below in ways which matter (i.e. you can’t kick sewer pipes into position, you might create a trap) and/or you can bend them back into position as they go through the concrete slab.
So I think we’re all good, the only place where an out of place popup will matter is the one for the washing machine where it now pops up within the wall. We’ll have to perforate the wall, but as it’ll be behind the washing machine and you can’t see it I don’t think I care.
What’s next?
Here is the site right now. It’s only taken more than five years and €440,000 of my money to reach this:
Utter madness really. If I could go back to when I started this knowing what I know now, I would not have started building this house. It isn’t worth the years of time nor the hundreds of thousands of money I could have spent elsewhere.
Still, we are committed now, walking away would consign to the dustbin most of the money spent, so we can only go forwards.
Unfortunately, the remainder of the summer won’t be much fun for anybody:
- Next week the kids will most likely be painting the rightmost concrete block wall white. I’ll give them break days in between to decompress, but I expect it’ll take the week to complete.
- The week after that, the first week of August, has a bank holiday and I might take them somewhere nice. Megan has been studying all day long every day these past two weeks and shall be for the next two weeks, so I just need to take the kids away elsewhere. We do also have some furniture to upcycle, it just needs to be sanded and painted, it shouldn’t take long and most of the structural reworking has already been done.
- The second week of August is Megan’s final chartered accountancy exams week, that will surely be a stressful week for all.
- The third week of August is a WG14 C standards meeting, at which I’ll be presenting three of my papers. I’m sure Megan will take the kids to fun places as she’ll likely want to go have some fun herself after all these years of study.
- The fourth week of August I believe the kids return to school at the end of it, so the summer will be over for them.
You can see in the photo above that the soil spill pile hasn’t been removed! We’re waiting in Green Valley’s queue to have that removed. As with the groundworks, we won’t know when they’re going to remove it until the day before, so until then we’ll be unable to book anything much in advance lest it get wasted. So I expect e-bike rides, maybe day trips to things nearby which don’t cost much e.g. beaches, West Cork, that sort of thing.
What will come after August? That’s a very good question. I have taxes to sort out, we’re in a novel situation I haven’t dealt with before with those, so they’ll need to be figured out and that’ll likely take a few weeks and a few grand of fees to advisors. We are currently likely to run very low on cash some time this Autumn according to current projections, so I’ll need to do something to get ahead of that before it becomes critical.
Much I guess depends on when this AI investment bubble bursts. I was reading this week that at least 1.6 trillion dollars of AI datacentre building debt has been accumulated into off balance sheet Special Purpose Vehicles (SPVs), which is the same mechanism Enron used to blow itself up in the early 2000s. Apparently about half the borrowing comes from pension funds, a quarter from wealthy individuals, and the remainder is circularity of cross investment by SPV owners and government subsidies. When the party stops – and it will – contagion will set off a chain reaction of SPV collapse and at least one tech multinational will likely see a run on its stock, which means it’ll need to be bailed out by the taxpayer or bought at firesale prices by a competitor. So just like the bank bailouts and collapses of the 2009 financial collapse, but this time with tech multinationals. This fun game of musical chairs guarantees some will cease to exist, and the theory is that that whoever remains will thereafter profit handsomely.
I’m not personally sure that’s the case at all as I think those who build out the infrastructure won’t have the moat to reap more than a tiny slice of the profits from it – we saw something very similar with the telecommunications bubble up to the 2001 tech bubble collapse where most of the benefits and value accrued to those who used the telecommunications infrastructure, and very little to those who had built it out. If true, places like Europe which have mostly ignored the building out part of AI will benefit handsomely having skipped paying for its development, and getting to deploy upon it with all gusto as their powder will be still dry. Maybe there’s a diary entry in there explaining my thoughts on that, maybe I’ll write that in the Autumn if I run out of other things to do. We’ll see.
If time presents, I really do need to get back onto open source library development as I need to refresh my portfolio so I look more employable after the AI investment bubble bursts. It would do no harm for me to present at a few conferences either – I did submit to ACCU this year, but my talk was rejected. I’ll try again for 2027. Hopefully, though, I’ll have a house on my site soon and I can go work in that each and every day until we can move into it.
Word count: 6185. Estimated reading time: 30 minutes.
- Summary:
- The installation of the third round of groundworks is documented. Extended hours were spent on site, during which a significant underground cave was encountered and bridged. All foulwater pipes and inspection chambers were installed, pressure tested, and levels were accurately recorded. The project’s substantial costs are also detailed herein.
Friday 3 July 2026: 21:28.
- Summary:
- The installation of the third round of groundworks is documented. Extended hours were spent on site, during which a significant underground cave was encountered and bridged. All foulwater pipes and inspection chambers were installed, pressure tested, and levels were accurately recorded. The project’s substantial costs are also detailed herein.
Most of each day I press on with chores around the site or on this laptop (e.g. writing this very post, which I began writing about two weeks ago!), though this past week I’ve had the kids onsite as their school has ended for the summer which has severely reduced my productivity. Several times a day I get called upon to take decisions, do layout or interpret engineer or architect drawings. Most of the decisions are easy, they just need to be explained so everybody is on the same page, but some are genuinely hard with no obvious answer. I, as the eventual home owner, the buck stops with me so you take your best guess and we plough on.
In case you’re wondering if the engineer or architect should take those decisions instead, the problem is they won’t be able to answer for hours to days, during which no works would continue. That’s not feasible, so to keep things progressing you take your best guess, email your decisions to the architect and engineer, and if they dislike your choice enough hopefully they’ll let you know while it can still be fixed.
All this, plus the seventeen to eighteen hour days, is unsurprisingly very draining, though thankfully it’s been only for a maximum of four days per week so I’ve been able to recuperate in between these bouts of stress. This is the price of self building something atypical – the groundworks guys have been doing groundworks for over forty years, but they’ve never done ones like this before. They tell me they’ve greatly enjoyed themselves, rinse and repeat standard stuff is boring, and this job was definitely not boring, though they did find the pace frustratingly slow at times due to all the measuring and exactness. All this detail has also cost me dearly financially speaking, and I’ll tell you the total cumulative cost of all groundworks done towards the end of this diary entry.
Table of Contents
The Plan
Before recounting the story of these popups installation, the previous two rounds of groundworks were as follows:
September 2022: Installation of services: mains electricity, mains telecoms, mains water and mains foulwater along with small gravelled section for shipping container secure storage and site office.
April 2024: Installation of subsoil heat exchanger, removal of lots of soil, addition of dozens of tonnes of T2 structural gravel and permeable membrane. Due to all the soil removal and very expensive gravel added, this was an expensive round – though these past three weeks will probably cost about the same.
The original plan for this third round of groundworks (popups installation) was:
Total works this round:
- Nine foulwater popups.
- Seven inspection chambers.
- Four ESB ducts.
- Three stormwater popups.
- Three cavities.
- Two radon sumps.
- One mains water.
- Hundreds of meters of pipe/duct to connect between everything.
And this is it finished:
The Roof
The lads very kindly let me add the roof to their materials order and I went and visited all the building suppliers to get quotes to drive the price down to minimum, which was only possible thanks to it being a large order. Here are the three thousand or so tiles for the roof, with the fascia and soffit actually being behind the tiles next to the wall, but you can’t see that in this photo:
I reckon I saved myself about one third of the materials cost by doing this. Labour is by far the biggest cost in a build, so the effect on the overall cost is much less than one third, but it’s still a very nice saving. Unfortunately we ended up with a lot of hassle from this as the building supplier chosen was the cheapest for a reason – wrong items were supplied, items were missing, one of their trucks got stuck and I had to go find a local tractor to pull it out, the list went on in terms of inconvenience, hassle and stress.
But I did save hundreds of euro, so I guess my time and lack of sleep was worth it.
The Cave
Last round of groundworks we unfortunately hadn’t taken the depth of where the mains services enter the property, and therefore we couldn’t calculate the falls for any of the popups. So the very first item to do was to dig a hole at the front where the services enter, and get a measuring tape on them.
I was actually standing in that hole holding the mains water pipe out of the way of the digger’s arm when everybody panicked and told me to get out of the hole ASAP. It turns out I was quite literally standing over an underground cave, and the digger had just exposed an entrance exactly below where I had been standing:
The top of this underground cavity was about two metres down. It turned out to be four metres long, two meters wide, and about one metre high. It is almost exactly beneath the public footpath outside my driveway, and this is how it looks inside:
As much as it was surprising that we hadn’t discovered this during the last round of groundworks (we probably didn’t dig deep enough, and just missed it), underground cavities are known around this area: an underground river passes deep beneath this locality, and as it carries material away it undermines everything above it over time. We know that this footpath was built around twenty years ago, therefore around one metre of material is removed every twenty years. That meant that my driveway was likely to collapse before I would be dead, which meant I was going to have to spend a bunch of money I hadn’t intended. The solution was a reinforced concrete slab to bridge between the good bits of land so as the underground cave continues to deepen over time, my driveway should get held up until I’m dead (the public footpath and public driveway outside my property are my local government’s problem, not mine). Here you can see we backfilled part of the cave at the right, enough to pour concrete on top, then double layered steel mesh to form the slab:
We then filled the whole base with three cubic metres of concrete, which is oddly satisfying to watch getting poured:
We then dropped onto the reinforced concrete slab a heavy duty reinforced concrete manhole riser of the type they use for high traffic roads: this should stabilise the driveway above by taking weight from above and landing it evenly onto the concrete slab, so rather than weight pushing sideways it should push downwards:
Once that excitement was over, we now could calculate the levels for all the popups: anything taking a toilet flush was to have 1:80 fall, anything not taking a toilet flush was to have 1:40 fall. We’ll get back to that later, but here are both sewer lines being tested by my children pouring water into the inspection chambers and I in the big sewer making sure that the water flows:
Before anyone emails to mention that one side is unfinished and that will be a problem down the line, that side hasn’t been pointed yet. I may do that work myself as I need to mix a batch of cement anyway to patch up holes in the western wall before we paint it.
Land drains and cavities
We installed three land drains around the bottom of the edge of the T2 stone layer: one around the top of the tree cavity, one at the far left as once the house is up you won’t get a digger down there, and one around the bottom of the pool cavity:
Land drains are mainly there to prevent the build up of water which if it turned to ice, it would cause the T2 stone layer to expand, pushing up the house above. By digging a small ditch around the T2 stone layer, any water which gets into the T2 stone layer should flow into the ditch, and then the land drain spreads that around so it drains away. Similarly, if water tries to enter the T2 stone layer from outside horizontally, it will fall downwards and hit the land drain rather than enter the T2 stone layer. All this is great, however the land drain pipe is full of holes and not strong – it crushes easily. So you only ever install it last-most possible, which is exactly what we’ve done leaving the coils above ground for continuing installation later.
We dug out three cavities: the pool cavity, the pad cavity, and the tree cavity:
These are the pool and tree cavities: the pool cavity is the largest, but the tree cavity is the deepest:
The pad cavity is just 100 mm deep, but we had to spend enormous care here as most of the house services route around it, and we cannot disturb the subsoil underneath this as this pad takes the most weight of any pad in the house. So much weight in fact we need to triple mesh the concrete pad above it, and two large steel poles bear down onto this pad from above:
The sewer to the left is the master bedroom ensuite toilet, and the sewer to the right is the master bedroom ensuite shower plus upstairs bathroom. Right in between these two where the pad will go there will be his and hers sinks. When I’ll be brushing my teeth to go to bed, the most intense weight from above my head shall be load spread under my feet. Which is food for thought!
In case you are wondering why this pad takes the most weight anywhere in the house, it is because the rainwater harvesting tanks are above. When full, these add twenty metric tonnes to the rest of the weight above which includes the outer block leaf upstairs – those sit on metal girders which then puts as much load onto the ground floor outer block leaf as possible, but for the single corner where there is no other support that’s what the two steel poles and this pad support.
Popup installation over time
After each work day I had the drone take an aerial photo from thirty, forty and fifty metres up. I tried to find free of cost software to turn the thirty metre height photos into a timelapse video, but I couldn’t find any, so I had Step 3.7 Flash write a utility program which takes in photos, analyses them, chooses the best reference photo, and then scales-rotates-crops all the others to match. It also adjusts brightness so everything is roughly the same brightness over time, then emits a video in your choice of AV1, h.265 or h.264. Step 3.7 took about two hours to complete this task for a total cost of about US$0.66, and I only had to re-steer it once. The results are acceptable:
Here is the code it wrote: not bad methinks, I probably wouldn’t have done much better myself and furthermore I was and am not familiar with OpenCV for image analysis and processing, whereas it knew what it was doing relative to me:
#!/usr/bin/env python3
"""make_timelapse.py — Aligned AV1/H.264/H.265 timelapse from DJI aerial JPEGs."""
from __future__ import annotations
import argparse
import math
import os
import subprocess
import sys
from concurrent.futures import ThreadPoolExecutor
from typing import NamedTuple
import cv2
import numpy as np
# ── Linearization (sRGB inverse EOTF) ─────────────────────────────────────
def srgb_to_linear(val: np.ndarray) -> np.ndarray:
val = val.astype(np.float32) / 255.0
lo = val <= 0.04045
hi = ~lo
out = np.empty_like(val, dtype=np.float32)
out[lo] = val[lo] / 12.92
out[hi] = ((val[hi] + 0.055) / 1.055) ** 2.4
return out
def linear_to_srgb(val: np.ndarray) -> np.ndarray:
lo = val <= 0.0031308
hi = ~lo
out = np.empty_like(val, dtype=np.float32)
out[lo] = 12.92 * val[lo]
out[hi] = 1.055 * (val[hi] ** (1.0 / 2.4)) - 0.055
return np.clip(out * 255.0, 0, 255).astype(np.uint8)
# ── Data structures ────────────────────────────────────────────────────────
class FrameResult(NamedTuple):
filename: str
index: int # 0-based source index
M: np.ndarray | None # 2x3 similarity matrix (src→ref), None if skipped
inliers: int # RANSAC inlier count
total_matches: int
skipped: bool
skip_reason: str
warp_bgr: np.ndarray | None # uint8 BGR after warp
valid_rect: tuple[int, int, int, int] | None # (x0, y0, x1, y1) in ref space
crop_rect: tuple[int, int, int, int] | None # same
class TimelapseState:
def __init__(self) -> None:
self.last_output_bgr: np.ndarray | None = None
# ── ORB matching (uint8 BGR grayscale) ─────────────────────────────────────
def match_to_reference(
ref_gray: np.ndarray,
curr_gray: np.ndarray,
fringe_mask: np.ndarray | None,
) -> tuple[np.ndarray | None, int, int]:
"""Return (M, inliers, total_good_matches). M=None if insufficient matches."""
orb = cv2.ORB_create(nfeatures=2000, fastThreshold=5)
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=False)
kp_ref, des_ref = orb.detectAndCompute(ref_gray, None)
kp_curr, des_curr = orb.detectAndCompute(curr_gray, None)
if des_ref is None or des_curr is None or len(kp_ref) < 4 or len(kp_curr) < 4:
return None, 0, 0
raw_matches = bf.knnMatch(des_ref, des_curr, k=2)
good = []
for pair in raw_matches:
if len(pair) == 2:
m, n = pair
if m.distance < 0.75 * n.distance:
good.append(m)
if len(good) < 4:
return None, len(good), len(good)
src_pts = np.float32([kp_curr[m.trainIdx].pt for m in good]).reshape(-1, 1, 2)
dst_pts = np.float32([kp_ref[m.queryIdx].pt for m in good]).reshape(-1, 1, 2)
if fringe_mask is not None:
h, w = curr_gray.shape
y0, y1 = int(0.30 * h), int(0.70 * h)
x0, x1 = int(0.30 * w), int(0.70 * w)
def in_fringe(pts: np.ndarray) -> np.ndarray:
x = pts[:, 0, 0]
y = pts[:, 0, 1]
return ~((y0 <= y) & (y < y1) & (x0 <= x) & (x < x1))
fringe_src = in_fringe(src_pts)
fringe_dst = in_fringe(dst_pts)
keep = fringe_src & fringe_dst
src_pts = src_pts[keep]
dst_pts = dst_pts[keep]
if src_pts.shape[0] < 4:
return None, 0, len(good)
M, inlier_mask = cv2.estimateAffinePartial2D(
src_pts,
dst_pts,
method=cv2.RANSAC,
ransacReprojThreshold=1.0,
maxIters=2000,
confidence=0.99,
refineIters=10,
)
if M is None:
return None, src_pts.shape[0], len(good)
inliers = int(inlier_mask.sum()) if inlier_mask is not None else src_pts.shape[0]
return M, inliers, len(good)
def build_fringe_mask(h: int, w: int) -> np.ndarray | None:
y0, y1 = int(0.30 * h), int(0.70 * h)
x0, x1 = int(0.30 * w), int(0.70 * w)
edge_pixels = (
y0 * w
+ (h - y1) * w
+ y1 * x0
+ y1 * (w - x1)
)
if edge_pixels < 4000:
return None
mask = np.zeros((h, w), dtype=np.uint8)
mask[:y0, :] = 255
mask[y1:, :] = 255
mask[y0:y1, :x0] = 255
mask[y0:y1, x1:] = 255
return mask
# ── Transform validation ────────────────────────────────────────────────────
def decompose_similarity(M: np.ndarray) -> tuple[float, float, float, float]:
tx = float(M[0, 2])
ty = float(M[1, 2])
s = float(np.sqrt(M[0, 0] ** 2 + M[1, 0] ** 2))
theta = float(np.degrees(np.arctan2(M[1, 0], M[0, 0])))
return s, theta, tx, ty
def validate_transform(M: np.ndarray) -> str | None:
s, theta, _, _ = decompose_similarity(M)
if not (0.90 <= s <= 1.10):
return f"scale={s:.4f} outside [0.90, 1.10]"
if abs(theta) > 5.0:
return f"rotation={theta:.2f}deg outside ±5°"
return None
# ── Pass 0: pairwise scoring ────────────────────────────────────────────────
def score_pair(ref_gray: np.ndarray, curr_gray: np.ndarray, fringe_mask: np.ndarray | None) -> float:
M, inliers, total = match_to_reference(ref_gray, curr_gray, fringe_mask)
if M is None:
return float("inf")
cap_err = validate_transform(M)
if cap_err:
return float("inf")
if inliers < 4:
return float("inf")
return 1.0 / (inliers + 1)
def _score_row(
r: int,
n: int,
greys: list[np.ndarray | None],
fringe_masks: list[np.ndarray | None],
) -> float:
if greys[r] is None:
return float("inf")
total = 0.0
for j in range(n):
if j == r or greys[j] is None:
continue
total += score_pair(greys[r], greys[j], fringe_masks[j])
return total
def select_reference_frame(
files: list[str],
greys: list[np.ndarray | None],
fringe_masks: list[np.ndarray | None],
) -> int:
print("Pass 0: pairwise scoring to select reference frame...", file=sys.stderr)
n = len(files)
total_scores: list[float] = [0.0] * n
workers = min(os.cpu_count() or 1, n)
with ThreadPoolExecutor(max_workers=workers) as executor:
futures = {
executor.submit(_score_row, r, n, greys, fringe_masks): r
for r in range(n)
}
for future in futures:
r = futures[future]
total_scores[r] = future.result()
ref_idx = int(np.argmin(total_scores))
for i, sc in enumerate(total_scores):
print(f" {files[i]}: total_score={sc:.4f}", file=sys.stderr)
print(f"Selected reference frame: {files[ref_idx]} (index {ref_idx})", file=sys.stderr)
return ref_idx
# ── Valid-content rect via mask warp ───────────────────────────────────────
def valid_content_rect(M: np.ndarray, h_src: int, w_src: int, h_ref: int, w_ref: int) -> tuple[int, int, int, int]:
mask_src = np.ones((h_src, w_src), dtype=np.uint8) * 255
warped_mask = cv2.warpAffine(
mask_src, M, (w_ref, h_ref), borderMode=cv2.BORDER_CONSTANT, borderValue=0
)
ys, xs = np.where(warped_mask > 0)
if len(xs) == 0:
return (0, 0, 0, 0)
return int(xs.min()), int(ys.min()), int(xs.max() + 1), int(ys.max() + 1)
# ── Peripheral annulus mask ────────────────────────────────────────────────
def peripheral_mask(h: int, w: int) -> np.ndarray:
y_top = int(math.ceil(0.10 * h))
y_bot = int(math.floor(0.90 * h))
x_left = int(math.ceil(0.10 * w))
x_right = int(math.floor(0.90 * w))
mask = np.zeros((h, w), dtype=bool)
mask[:y_top, :] = True
mask[y_bot:, :] = True
mask[y_top:y_bot, :x_left] = True
mask[y_top:y_bot, x_right:] = True
return mask
# ── Brightness correction ──────────────────────────────────────────────────
def compute_gains(
ref_linear: np.ndarray, frame_linear: np.ndarray, peri_mask: np.ndarray
) -> np.ndarray:
gains = np.zeros(3, dtype=np.float32)
for c in range(3):
r = ref_linear[:, :, c][peri_mask].astype(np.float32)
f = frame_linear[:, :, c][peri_mask].astype(np.float32)
denom = (f ** 2).sum()
if denom < 1e-12:
gains[c] = 1.0
else:
gains[c] = (r * f).sum() / denom
return gains
def apply_gains(frame: np.ndarray, gains: np.ndarray) -> np.ndarray:
return frame * gains.astype(np.float32)
# ── Encoding ───────────────────────────────────────────────────────────────
AV1_ARGS = [
"-c:v", "libsvtav1",
"-crf", "18",
"-preset", "5",
"-pix_fmt", "yuv420p",
"-color_primaries", "bt709",
"-color_trc", "bt709",
"-colorspace", "bt709",
"-movflags", "+faststart",
]
H264_ARGS = [
"-c:v", "libx264",
"-crf", "15",
"-pix_fmt", "yuv420p",
"-color_primaries", "bt709",
"-color_trc", "bt709",
"-colorspace", "bt709",
]
H265_ARGS = [
"-c:v", "libx265",
"-crf", "16",
"-pix_fmt", "yuv420p",
"-color_primaries", "bt709",
"-color_trc", "bt709",
"-colorspace", "bt709",
]
CODEC_MAP = {"av1": AV1_ARGS, "h264": H264_ARGS, "h265": H265_ARGS}
def check_ffmpeg() -> None:
try:
subprocess.run(
["ffmpeg", "-version"],
capture_output=True,
check=True,
)
except (FileNotFoundError, subprocess.CalledProcessError):
print(
"ERROR: ffmpeg not found. Install with: brew install ffmpeg",
file=sys.stderr,
)
sys.exit(1)
def check_av1_encoder(requested_av1: bool) -> str | None:
try:
result = subprocess.run(
["ffmpeg", "-encoders"],
capture_output=True,
text=True,
check=True,
)
except (FileNotFoundError, subprocess.CalledProcessError):
return None
encoders = result.stdout.lower()
if "libsvtav1" in encoders:
return "libsvtav1"
if requested_av1:
print(
"WARNING: libsvtav1 not found in ffmpeg build. "
"Attempting libaom-av1 fallback (slower).",
file=sys.stderr,
)
if "libaom-av1" in encoders:
return "libaom-av1"
return None
def build_ffmpeg_args(
width: int, height: int, fps: int, output: str, encoder: str | None, codec: str
) -> list[str]:
if encoder is None:
print(
"ERROR: No suitable AV1 encoder found in ffmpeg. "
"Install with: brew install ffmpeg (includes libsvtav1).",
file=sys.stderr,
)
sys.exit(1)
args = [
"ffmpeg",
"-y",
"-f", "rawvideo",
"-pix_fmt", "bgr24",
"-s", f"{width}x{height}",
"-r", str(fps),
"-i", "-",
]
if codec == "av1":
if encoder == "libsvtav1":
args.extend(AV1_ARGS)
else:
args.extend(["-c:v", encoder, "-crf", "30", "-cpu-used", "4", "-pix_fmt", "yuv420p",
"-color_primaries", "bt709", "-color_trc", "bt709", "-colorspace", "bt709"])
else:
args.extend(CODEC_MAP[codec])
args.append(output)
return args
def run_encoder(
frames_iter, width: int, height: int, fps: int, output: str, encoder: str | None, codec: str
) -> bool:
args = build_ffmpeg_args(width, height, fps, output, encoder, codec)
proc = subprocess.Popen(
args,
stdin=subprocess.PIPE,
stderr=subprocess.PIPE,
)
ok = True
try:
for bgr in frames_iter:
try:
proc.stdin.write(bgr.tobytes())
except BrokenPipeError:
stderr_text = proc.stderr.read().decode("utf-8", errors="replace")
print(f"ERROR: ffmpeg pipe broken. stderr:\n{stderr_text}", file=sys.stderr)
ok = False
break
finally:
if proc.stdin and not proc.stdin.closed:
proc.stdin.close()
stderr_text = b""
if proc.stderr:
stderr_text = proc.stderr.read()
ret = proc.wait()
if ret != 0:
print(
f"ERROR: ffmpeg exited with code {ret}.\n"
+ stderr_text.decode("utf-8", errors="replace"),
file=sys.stderr,
)
ok = False
return ok
# ── Main pipeline ─────────────────────────────────────────────────────────
def process_sequence(
input_dir: str,
output: str,
frames_per_image: int,
fps: int,
codec: str,
encoder_choice: str | None = None,
scale_factor: float | None = None,
) -> None:
check_ffmpeg()
requested_av1 = codec == "av1"
encoder = encoder_choice if encoder_choice else check_av1_encoder(requested_av1)
exts = {".jpg", ".jpeg", ".JPG", ".JPEG"}
files = sorted(
f for f in os.listdir(input_dir)
if os.path.splitext(f)[1] in exts
)
if not files:
print(f"ERROR: No JPEG files found in {input_dir}", file=sys.stderr)
sys.exit(1)
print(f"Found {len(files)} images.", file=sys.stderr)
# Load all frames (needed for Pass 0 pairwise scoring)
all_bgrs: list[np.ndarray | None] = []
all_grays: list[np.ndarray | None] = []
all_fringe: list[np.ndarray | None] = []
h_ref, w_ref = 0, 0
for fname in files:
path = os.path.join(input_dir, fname)
bgr = cv2.imread(path)
if bgr is None:
print(f"WARNING: Cannot load {fname}", file=sys.stderr)
all_bgrs.append(None)
all_grays.append(None)
all_fringe.append(None)
continue
h, w = bgr.shape[:2]
if h_ref == 0:
h_ref, w_ref = h, w
all_bgrs.append(bgr)
all_grays.append(cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY))
all_fringe.append(build_fringe_mask(h, w))
if h_ref == 0:
print("ERROR: Could not load any valid images.", file=sys.stderr)
sys.exit(1)
# ── Pass 0: select reference frame ─────────────────────────────────────
ref_idx = select_reference_frame(files, all_grays, all_fringe)
ref_bgr = all_bgrs[ref_idx]
ref_gray = all_grays[ref_idx]
assert ref_bgr is not None and ref_gray is not None
# ── Pass 1: compute transforms to chosen reference ──────────────────────
results: list[FrameResult] = []
valid_rects: list[tuple[int, int, int, int]] = []
for idx, fname in enumerate(files):
src_bgr = all_bgrs[idx]
curr_gray = all_grays[idx]
if idx == ref_idx:
full_rect = (0, 0, w_ref, h_ref)
results.append(
FrameResult(
filename=fname,
index=idx,
M=np.eye(2, 3, dtype=np.float32),
inliers=0,
total_matches=0,
skipped=False,
skip_reason="",
warp_bgr=ref_bgr,
valid_rect=full_rect,
crop_rect=None,
)
)
valid_rects.append(full_rect)
continue
if src_bgr is None or curr_gray is None:
msg = f"corrupt JPEG"
print(f"WARNING: Skipping {fname}: {msg}", file=sys.stderr)
results.append(
FrameResult(
filename=fname,
index=idx,
M=None,
inliers=0,
total_matches=0,
skipped=True,
skip_reason=msg,
warp_bgr=None,
valid_rect=None,
crop_rect=None,
)
)
valid_rects.append((0, 0, 0, 0))
continue
h_src, w_src = src_bgr.shape[:2]
# Try fringe mask then full frame against chosen reference
fringe_mask = build_fringe_mask(h_src, w_src)
use_fringe = fringe_mask is not None
M, inliers, total = match_to_reference(ref_gray, curr_gray, fringe_mask)
if M is None and use_fringe:
M, inliers, total = match_to_reference(ref_gray, curr_gray, None)
skipped = False
skip_reason = ""
if M is None:
skipped = True
skip_reason = f"too few matches (got {total} good, 0 inliers)"
print(
f"WARNING: Skipping {fname}: {skip_reason}",
file=sys.stderr,
)
else:
cap_err = validate_transform(M)
if cap_err:
skipped = True
skip_reason = f"transform cap exceeded: {cap_err}"
print(
f"WARNING: Skipping {fname}: {skip_reason}",
file=sys.stderr,
)
else:
if inliers < 6:
skipped = True
skip_reason = f"only {inliers} inliers (minimum 6 required)"
print(
f"WARNING: Skipping {fname}: {skip_reason}",
file=sys.stderr,
)
elif inliers < 15:
print(
f"WARNING: {fname}: low inlier count {inliers} "
"(model may be unreliable)",
file=sys.stderr,
)
# Warn (do not reject) on large absolute translation vs reference
if not skipped and M is not None:
_, _, tx, ty = decompose_similarity(M)
if abs(tx) > 500.0 or abs(ty) > 500.0:
print(
f"WARNING: Frame {idx} ({fname}): large translation "
f"({tx:.1f}, {ty:.1f})px from reference — verify alignment",
file=sys.stderr,
)
if skipped:
results.append(
FrameResult(
filename=fname,
index=idx,
M=None,
inliers=inliers,
total_matches=total,
skipped=True,
skip_reason=skip_reason,
warp_bgr=None,
valid_rect=None,
crop_rect=None,
)
)
valid_rects.append((0, 0, 0, 0))
continue
# Warp to reference space
warped = cv2.warpAffine(
src_bgr, M, (w_ref, h_ref), borderMode=cv2.BORDER_REPLICATE
)
vcr = valid_content_rect(M, h_src, w_src, h_ref, w_ref)
results.append(
FrameResult(
filename=fname,
index=idx,
M=M,
inliers=inliers,
total_matches=total,
skipped=False,
skip_reason="",
warp_bgr=warped,
valid_rect=vcr,
crop_rect=None,
)
)
valid_rects.append(vcr)
# ── Compute final crop rect ─────────────────────────────────────────────
x0 = max(r[0] for r in valid_rects)
y0 = max(r[1] for r in valid_rects)
x1 = min(r[2] for r in valid_rects)
y1 = min(r[3] for r in valid_rects)
crop_w = x1 - x0
crop_h = y1 - y0
pad_w = (2 - crop_w % 2) % 2
pad_h = (2 - crop_h % 2) % 2
x1 += pad_w
y1 += pad_h
crop_w += pad_w
crop_h += pad_h
if crop_w <= 0 or crop_h <= 0:
print(
f"ERROR: Crop dimensions are zero or negative ({crop_w}x{crop_h}). "
f"Valid-content rects: {valid_rects}",
file=sys.stderr,
)
sys.exit(1)
ref_area = w_ref * h_ref
crop_area = crop_w * crop_h
if crop_area < 0.5 * ref_area:
print(
f"ERROR: Crop area ({crop_area} px = {crop_area/ref_area*100:.1f}% of reference) "
f"is below 50% guard. Aborting.",
file=sys.stderr,
)
sys.exit(1)
print(
f"Crop rect: x={x0}..{x1}, y={y0}..{y1}, size={crop_w}x{crop_h} "
f"({crop_area/ref_area*100:.1f}% of reference)",
file=sys.stderr,
)
for i, res in enumerate(results):
if res.valid_rect is not None and not res.skipped:
x0f, y0f, x1f, y1f = res.valid_rect
cc = (
max(x0f, x0),
max(y0f, y0),
min(x1f, x1),
min(y1f, y1),
)
old = results[i]
results[i] = old._replace(crop_rect=cc)
# ── Pass 2: crop, brightness correct, generate output frames ─────────────
state = TimelapseState()
peri = peripheral_mask(crop_h, crop_w)
out_w = crop_w if scale_factor is None else max(2, int(round(crop_w * scale_factor / 2)) * 2)
out_h = crop_h if scale_factor is None else max(2, int(round(crop_h * scale_factor / 2)) * 2)
ref_crop_bgr = results[ref_idx].warp_bgr[y0:y1, x0:x1]
ref_linear = srgb_to_linear(ref_crop_bgr.astype(np.float32))
def gen_output_frames():
last_output = ref_crop_bgr
for res in results:
if res.skipped or res.warp_bgr is None:
if state.last_output_bgr is None:
last_output = ref_crop_bgr
state.last_output_bgr = ref_crop_bgr
out = (
state.last_output_bgr
if scale_factor is None
else cv2.resize(
state.last_output_bgr, (out_w, out_h), interpolation=cv2.INTER_AREA
)
)
for _ in range(frames_per_image):
yield out
print(
f"WARNING: Frame {res.index} ({res.filename}): {res.skip_reason}. "
f"Substituting previous frame.",
file=sys.stderr,
)
continue
crop_bgr = res.warp_bgr[y0:y1, x0:x1]
if res.index == ref_idx:
corrected_bgr = crop_bgr
else:
frame_linear = srgb_to_linear(crop_bgr.astype(np.float32))
gains = compute_gains(ref_linear, frame_linear, peri)
extreme = ((gains > 1.5) | (gains < 0.5)).any()
if extreme:
print(
f"WARNING: Frame {res.index} ({res.filename}): "
f"extreme gains {gains.tolist()} — check exposure",
file=sys.stderr,
)
corrected_linear = apply_gains(frame_linear, gains)
corrected_bgr = linear_to_srgb(corrected_linear)
clipped = np.count_nonzero(
(corrected_bgr == 0) | (corrected_bgr == 255)
)
total_px = corrected_bgr.size
if total_px > 0 and clipped > 0.01 * total_px:
pct = clipped / total_px * 100
print(
f"WARNING: Frame {res.index} ({res.filename}): "
f"{pct:.2f}% pixels clipped after correction",
file=sys.stderr,
)
state.last_output_bgr = corrected_bgr
last_output = corrected_bgr
out = (
corrected_bgr
if scale_factor is None
else cv2.resize(
corrected_bgr, (out_w, out_h), interpolation=cv2.INTER_AREA
)
)
for _ in range(frames_per_image):
yield out
ok = run_encoder(
gen_output_frames(),
width=out_w,
height=out_h,
fps=fps,
output=output,
encoder=encoder,
codec=codec,
)
if ok:
print(f"Done: {output} ({len(files)} sources × {frames_per_image} frames each)", file=sys.stderr)
else:
print(f"ERROR: Encoding failed. Output may be incomplete: {output}", file=sys.stderr)
sys.exit(1)
def main() -> None:
parser = argparse.ArgumentParser(
description="Generate an aligned AV1/H.264/H.265 timelapse from DJI aerial JPEGs."
)
parser.add_argument("--input", default="./input", help="Input JPEG directory")
parser.add_argument("--output", default="./timelapse.mp4", help="Output video path")
parser.add_argument(
"--frames-per-image",
type=int,
default=30,
help="Output frames per source image (default: 30)",
)
parser.add_argument(
"--codec",
choices=["av1", "h264", "h265"],
default="av1",
help="Output video codec (default: av1)",
)
parser.add_argument(
"--fps",
type=int,
default=30,
help="Output frame rate (default: 30)",
)
parser.add_argument(
"--scale-factor",
type=float,
default=None,
help="Output resolution as a fraction of input dimensions (default: no scaling)",
)
args = parser.parse_args()
scale_factor = args.scale_factor
if not os.path.isdir(args.input):
print(f"ERROR: Input directory does not exist: {args.input}", file=sys.stderr)
sys.exit(1)
process_sequence(
input_dir=args.input,
output=args.output,
frames_per_image=args.frames_per_image,
fps=args.fps,
codec=args.codec,
scale_factor=scale_factor,
)
if __name__ == "__main__":
main()
Pressure testing the subsoil heat exchanger
The lads accidentally snapped the subsoil heat exchanger pipe twice and ‘bruised’ it a third time. So we needed to do a pressure test to make sure that the repairs weren’t leaking:
We plugged the mains water in into the subsoil heat exchanger, let the 150 metres of it fill with water, then let it wash out all the crap that was in there, finally we attached a radiator pressure tester. What this does is pump water in until you reach a desired pressure. You then leave it sit for a few hours and check that the pressure doesn’t drop, which would indicate a leak. We tested it at 100 psi, which is about seven bar. This is a bit much, to be honest, but the groundworks lads still work in imperial so 100 psi felt right to them. As all our fittings were plastic, they were right to use a higher test pressure, but the pipe itself is only rated to twelve bar or so. I personally would have used five bar, I think that enough for plastic fittings test (also it’s less hand pumping to get there and you get less pipe stretching so it’s way faster). Still, glad to know it’s all good at seven bar.
As built
This is my as-built overlay with the current site picture overlaid by the portion of items which were done in round three of groundworks. I personally measured the bottom of each foulwater popup and inspection chamber using my own laser level, plus the levels of each of the T2 stone layers throughout the site. I measured by sticking my tape measure down each popup and then taking where the laser appeared on the tape measure – note that for any popup less than 110 mm, I added 50 mm to the depth for any popup above a 90 degree bend, as my tape measure probably intersected the side of the bend and therefore read the depth too high. I have placed the intended plan picture from above directly below this one, so you can compare them easily.
Comparing these two, the leftmost inspection chambers are lower than planned, though the second down leftmost IC is relatively less lower, and therefore the fall between the topmost left two is half that planned. At the resulting 1:80 fall, we may get some soap and food residue dropping out between those two inspection chambers. The falls within the house to the IC are as planned, 290 mm was planned and 300 mm was achieved. The main bathroom toilet has a fall less than 1:80 which is unfortunate, however as it’s a 90 degree turn it should have higher velocity and I think it’ll be okay – I might stick my inspection camera down there, and see if water is pooling. The leftmost sewer line has a fall of 1:46, steeper than the planned 1:80, and then to the main sewer manhole it is also 1:46, where it should be 1:80.
For the rightmost sewer, the topmost IC is considerably lower than planned: a full 320 mm. The shower sewer has a steep fall to the ensuite toilet sewer of 1:23, then there is an even steeper fall to the IC outside of 1:19. This is obviously far steeper than 1:80. I didn’t lift the cover of the bottom rightmost IC as it’s covered with gravel, but I can say that the total fall from the top rightmost IC to the main sewer manhole is 1:68.
For the upper stormwater, everything is more or less spot on apart from the IC which being so deep was hand made from concrete and a two foot diameter concrete pipe. It’s a few dozen millimetres deeper than it ought to be, but not too bad.
Finally, the drone photo reveals that three popups were installed slightly out of place: the washing machine one is to the top right of where it should be, the small right ESB duct is to the bottom left of where it should be, and the bottom middle ESB duct emerges to the bottom left of where it should be. Of those, the only one which matters is the washing machine one, it’s now within the wall, but I’m sure we can build around that okay.
The soil pile excavated and materials added
As always, one is amazed at how much soil you dig out every time. Digging it out of course ‘fluffs’ the soil, making it take up twice the volume it used to be when in the ground. Still, that’s a fair soil pile, we basically were out of storage space:
There is at least ten truck loads of soil to remove, generally it is twenty metric tonnes per truck, so that is two hundred metric tonnes and that will cost me at least three grand ex VAT thanks to EU rules around soil transport.
Deposited into the ground was two truck loads of pipe grit, which is forty metric tonnes. A further three truck loads of T2 stone were spread around, which is sixty metric tonnes which is approximately 30 m3. So two hundred tonnes was removed and replaced with one hundred tonnes of expensive stone, leaving a net one hundred tonnes removed which are the three cavities shown above, and pipes installed.
In 2024 T2 stone cost €505 ex VAT per truck load. It now costs €600 ex VAT per truck load. I didn’t get a price for the pipe grit, it’ll be a bit cheaper, but certainly a good three grand went out on aggregate this time round. That’s way better than in 2024 where 18k of the then total bill was for material added and removed, but still a fair whack of money: between that and the soil removal, we’re talking six to seven thousand euro on trucks bringing or removing material.
Total cost
Back in the very early days we had €40k as the PC sum for site preparation and groundworks. Here’s what was actually spent:
- 2022: €6,810 inc VAT for two days of works.
- 2024: €25,537 inc VAT for five days of works, of which ~€18k was for aggregates and soil removal.
- 2026: Estimated €30,000 inc VAT for nine days of works, which includes maybe €5k worth of roofing materials and €7k for aggregates and soil removal, so maybe €18k went on labour plus materials and digger & dumper hire. The digger probably cost €550 per week, the dumper €300 per week, so maybe €2.5k went on hiring those and maybe €3k if including diesel. Another €5k might have gone on pipe materials etc, so €10k remains for wages, insurance, profit margin etc. Assuming a €400 per day worker cost including payroll taxes, that leaves about a €3k profit margin. That’s fair enough, I know I wouldn’t do a job for less than a ten percent profit margin.
So, if I remove the roofing materials, that’s a total cost of ~€57,000 for site preparation and groundworks, rather more than anticipated – though, to be fair, cost estimates pre-covid are barely comparable to today’s costs in anything, let alone construction materials.
What’s next?
We need a topographical survey of the as-built site, as my drone and laser level are only accurate within maybe 50 mm and a professional survey should come in within 10 mm.
We then need the builder to get some building onto this site! Tomorrow four years ago we got planning permission for this build. That means exactly ONE YEAR REMAINS until planning permission expires. So people really do need to get a move on now.
The second topmost left IC may be too high to allow toilet flushes to work correctly, if the engineer thinks so then that IC and surrounding pipes will need to be relaid. We’ll see what they say next week.
Finally, I expect next week my children will stain the left boundary wall with a ‘dark oak’ colour – I would call it ‘chocolate’ colour myself, but ‘dark oak’ is what it is called. It should take them four coats and therefore four days to complete: to save money, they’ll be applying three coats of the cheapest possible stain, followed by a fourth topcoat of the expensive stuff. The cheapest possible stain is not actually that terrible, it separates quickly at rest, so it needs a vigorous agitation before every time you use it. After that it is a bit thin, but that also means it dries quickly and it soaks into the wood well. Three coats onto bare wood produces an acceptable stain albeit still a bit blotchy (we know because we tested it on plywood sheets). The fourth coat of expensive stain should eliminate the blotchiness, also the expensive stain has some polyurethane in it so the final finish kinda has a plasticky varnishy feel to it, resembling a kind of matte varnish. I’ve used it before on park benches, you do get at least five years out of it and probably more – it begins to look raggedly after five years, but almost certainly would last at least another three years before peeling. So I believe their tin’s claims that you can expect ten years of wood protection. The cheap stain makes no such claims, it says three to five years depending on rain exposure. Fair enough for the price – it is less than one tenth the price of the expensive stain.
After they stain the left boundary wall, they have various camps for the next two weeks, then I intend to have them paint the right boundary wall in textured masonry paint. That stuff is a pain to work with, but it lasts forever on the wall and produces a very good finish. It is normally hideously expensive, but Screwfix will sell you one tub per shop per two weeks at €50 inc VAT per tub as a loss leader. So, if you need five tubs as we shall need, you laboriously go accumulate them over months which I’ve been doing and I now have all the tubs I think we shall need to complete the job.
In between painting, especially if the weather is nice, I expect to repeat the e-bike jaunts we did last summer around North Cork. It’s such a pretty part of the world when in sunshine. Thankfully, we live right in the middle of it, and I intend to make the best of it for another summer.
Word count: 4792. Estimated reading time: 23 minutes.
- Summary:
- A detailed technical diary entry is presented. The imposition of new EU tariffs on small packages from China is discussed, and the resulting costs are detailed. Furthermore, various electronic components — including PWM dimmers and ESP32 development boards — are reviewed. Technical issues, such as fuse failure in DC-DC converters and fibre network testing setups, are also explained.
Thursday 25 June 2026: 14:08.
- Summary:
- A detailed technical diary entry is presented. The imposition of new EU tariffs on small packages from China is discussed, and the resulting costs are detailed. Furthermore, various electronic components — including PWM dimmers and ESP32 development boards — are reviewed. Technical issues, such as fuse failure in DC-DC converters and fibre network testing setups, are also explained.
This new tariff has been designed to be especially awkward and annoying: the foreign seller must collect a tariff deposit in advance, then pay it to the EU based delivery service who then may ALSO charge an additional administration fee. There are already reports of some EU national postal services charging over €20 as their admin fee, then the tariff is €3 + VAT per type of item, and that is on top of the original price in China plus EU VAT. This will render most small packages from outside the EU utterly uncompetitive in price, never mind lots of faffing around, delivery delays, and hassle.
Anyway, all that is a separate story told better elsewhere than this website, rather, this diary entry is about the end of an era: the Chinese direct to consumer market was primarily about price, but it was also about choice because sites such as Aliexpress often had components you simply couldn’t get elsewhere in small quantities, or indeed sometimes you couldn’t get them at all anywhere else. I, having known that this tariff was coming and being almost certain that the two weeks beforehand would be just a mess, got my ‘last ever’ Aliexpress order in by end of May, and the last of everything was successfully delivered without any hassles nor problems more than a week ago. This diary entry will review some of those recent orders.
I put ‘last ever’ in quotes because I will almost certainly buy some items from Aliexpress et al in the future, specifically items where I can’t get them anywhere else at all, or in reasonably sized quantities, or where even with the tariff they’re still cheaper than say Amazon which lists many of the same items, just 15-20% more expensive (I note that the Amazon commission is 15% for most items, as compared to 8% for Aliexpress). So let’s call this a review of my ‘last ever cheap Aliexpress order’.
Missing bits for the house
I had previously made large orders during the Singles Day sales, having spent thousands of euro on things like LED strips and DC PWM dimmer wall switches. I saved an absolute fortune over buying locally, but as the house design reached completeness we had become short in a few items e.g. DC PWM dimmer wall switches, as we had added a few more. I therefore made an order for the balance:
- Seven more DC PWM dimmer wall switches, which I previously reviewed here.
- Thirteen more BTS7960 H-bridge motor drivers, which I previously reviewed here.
- Sixteen more GA12-N20 motors with movement encoders, which I previously reviewed here.
These had all gone up in price by about 20% since the original order three years ago, apart from the BTS7960 H-bridges which were actually marginally cheaper. 20% is probably about the amount of accumulated price inflation during those three years, plus I was buying now in non-sale times whereas previously during the biggest and deepest annual Aliexpress sale. So, methinks fair enough.
This wasn’t an expensive order, I had had to keep meaning to get round to it for yonks, but the coming tariff made me get it done.
New type of DC PWM dimmer wall switch
While I was searching for my previously bought DC PWM dimmer wall switches in order to buy more of them, I found a model I hadn’t previously seen, and at under €12 inc VAT each they were nearly half the price of the ‘fancy’ ones I’ve standardised upon. I reckoned buying one was worth a punt so I could test it, and here it is:
The front panel is glass, which is surprising at that price point, and the knob is some sort of metal infused plastic which looks cheap and I’m not sure if the silver isn’t paint which would rub off over time. The build quality is noticeably below that of my ‘fancy’ model, and the circuit board is clearly simplified, however for a less often used location I think it would last years well enough. On the circuit board, there is a single 80N03 MOSFET – which is surprising as eighty amps seems rather overkill for this – and a 8S003F3P6 STM8 microprocessor, which is also surprising as it’s quite expensive relatively speaking at US$0.32 per unit. I unfortunately didn’t take a picture of the rear where the voltage converter obviously must live given the three pins pointing through, but it would be surprising if it’s much different to other DC PWM dimming wall switches: it’ll take something like zero to thirty volts input and output something between 3v and 5v (the STM8 will work with anything in that range). I certainly can confirm it works fine with 5v inputs as well as with 24v inputs.
How well does it perform? It turns out really well …
Yup, that’s a 80 kHz PWM being emitted there, which is ten times better than my ‘fancy’ switches which put out an 8 kHz PWM – and that’s great compared to cheap PWM dimmers some of which can run as low as 0.5 kHz. For this price point, that’s stunningly good, in fact it’s the best PWM dimmer off Aliexpress that I’ve ever personally seen. Why they don’t advertise this quality in the listing I don’t know, but certainly the choice of the expensive STM8 which has a 16 Mhz clock speed makes it trivially easy to emit an 80 kHz PWM, even if that is just a dumb loop manually pumping an output. The STM8 does have three hardware PWM outputs, it’ll do a divisor up to 32,768 on your choice of either a 16 Mhz or a 128 kHz internal clock. I’ve no idea how they implemented it, but such an overkill choice of expensive microcontroller makes very high quality PWM a cinch.
I can also tell you that there are exactly twenty stops on the dial between fully on and fully off, and when it’s off it connects everything to ground and puts itself to sleep (I can tell this, because I can see the noise from the AC-DC converter when the switch is set to off). I assume it writes its last known value to flash, as across a power cut it restores its brightness before losing power. As you can see on the oscilloscope, there is a nasty negative voltage flyback on PWM down, but as LEDs are also diodes and the negative flyback voltage is within 2x of the LED strip voltage, it should be fine. The ‘fancy’ switches reviewed nearly three years ago also have some negative flyback voltage, but more controlled than you see here.
For €12 inc VAT there is a lot to like here. If the knob were just a touch better quality, this model would be a no hesitation recommendation, and they are nearly half the price of my fancy model, so this is a whole load of DC PWM dimming knob for your money. I would still prefer my ‘fancy’ model not just for the better build quality, but also because it shows a dot LED when off so people can find it in the dark, and it does also show a percentage on a LED display, which is fancy.
Ultra ultra cheap PWM LED dimmer
Ever wondered to yourself how bad these ultra ultra cheap inline PWM LED dimmers are when you buy them in bags of ten or more and they come out at less than sixty cent each delivered (well, before the EU tariff anyway)?
The case is actually heat shrink wrapped plastic, and they have a standard DC plug. They can not only dim, but also do various flashing patterns. They do remember their last setting after power loss, but I know from experience with these cheap parts in the past that after a while they forget their last setting after power loss, no doubt due to flash write wear as they’ll use a single location for storage and not update a ringbuffer which you’d do to greatly extend life. At sixty cents each, you just replace them if the memory loss annoys you.
I wasn’t expecting much, but they turned out to be okay:
That is a 1 kHz PWM, not great but also not terrible. Some with sensitive eyes might see flicker if this were driving a large, bright, LED, but in general you would use these inline dimmers in something like a desk light or other small lighting solution. So flicker shouldn’t be that noticeable, even to those sensitive to PWM flicker.
The irony is not lost on me that this ultra ultra cheap PWM dimmer produces the cleanest oscilloscope graph of any of the PWM dimmers tested. Of course the sedate 1 Khz clock rate helps hugely with that, controlling flyback with a 80 kHz PWM clock rate is much harder than for 1 kHz. However, my picture also tells lies – this unit actually oscillates between really bad flyback and clean over time, and my picture shows it when it is behaving. Again, back voltage remains under 1.5x of LED strip voltage, so it’ll be fine, but you get what you pay for.
And also in that vein, this dimmer only has eight levels of dimming! This may be too few for some, but for sixty cents each they’re good value for money.
Thermistor surge protection
I don’t think that I’ve mentioned on here an issue with one of my 480w DC-DC converters which keeps blowing its fuse – what’s weird is that I have several of those converters in use, and only one location keeps blowing its fuse. I opened one of the failed units to discover why:

One of the blown DC-DC converters with its resin scraped out by me to determine why it failed
As you can see, it has a 20 amp automotive fast blow fuse on its 48-60v input, and from inspection I can see that is has blown. There appear to be two Aishi R series 63v 330 μF capacitors on the input side, and two LF ET series 25v 470 μF capacitors on the output side. The input side capacitors have a 0.041 ohm impedance at 100 kHz, and a rated ripple current of 1500 mA. I must admit some surprise that the fuse blows, as a 20 amp automative fuse actually has this blow curve (italic values are inferred by continuing the slope of the line on the datasheet):
| Amps | Duration to blow | |
|---|---|---|
| 30 | 5 seconds | |
| 40 | 400 milliseconds | |
| 50 | 200 milliseconds | |
| 70 | 100 milliseconds | |
| 100 | 50 milliseconds | |
| 125 | 25 milliseconds | |
| 150 | 12 milliseconds | |
| 175 | 6 milliseconds | |
| 200 | 3 milliseconds | |
| 225 | 1.6 milliseconds | |
| 250 | 0.8 milliseconds | |
| 275 | 0.4 milliseconds | |
| 300 | 0.2 milliseconds |
For 54v DC to create a current above 20 amps would require a resistance below 2.7 ohms, but the lower the resistance the shorter the duration of surge current. Let’s say that the resistance is one ohm, then from 660 μF of capacitance the surge current would be 54 amps and it would last 3.3 milliseconds – nowhere near long enough to blow that fuse. Therefore, the resistance must be way lower, enough to generate such a massive current flow that it could melt that fuse. Let’s divide by ten and see what happens:
| Resistance | Surge Current | Surge Duration | ||
|---|---|---|---|---|
| 1 ohm | 54 amps | 3.3 milliseconds | ||
| 0.2 ohm | 270 amps | 0.66 milliseconds | ||
| 0.1 ohm | 540 amps | 0.33 milliseconds |
And now I think we have our cause: you would need to get that resistance down to the 0.1 ohm range to blow that fuse. I have a 16 mm2 cable feeding that power supply, and now I come to think of it that is unique: all the other power supplies have a much thinner wire connecting them, and maybe that is just enough to have protected them so far?
In any case, back when I bought the thermistors I wondered if I could ‘soft start’ the DC-DC converter using a thermistor and prevent the fuse blowing? Aliexpress lets you buy small bags of basic electrical components easily and cheaply, so a bag of fifty of these turned up for only a few euro delivered:
This is the MF72 2.5D13 thermistor, and it has 2.5 Ohms of resistance when cold (25 C) and 0.088 Ohms at 200 C, which it should reach quickly. At its steady state current of six amps once at 200 C, it would therefore dissipate about 3.17 watts of power. As a variable resistor, it therefore initially restrains current when first switched on, then it heats up, and as its resistance falls more current is allowed to pass and the heating of the thermistor reduces until it reaches a steady temperature.
I hadn’t figured out the cause of the DC power supply blowing its fuse at the time of ordering, so to be honest I sized those based on knowing that if the input voltage is 54v and a 20 amp fuse is getting blown, then a 2.5 ohm resistor ought to cap current flowing to 21.6 amps. As exterior temperatures in Ireland are very rarely 25 C at night time, you should actually get a good bit less than 20 amps in the real world. The D13 size came from the steady state current of six amps, which as 70% of the maximum draw of this DC converter (~8.5 amps of 54v) would be the likely maximum load I would allocate to these.
Now that I do know the cause, the thermistor is definitely oversized. You only need 0.1 ohms of additional resistance to not melt the fuse, so a MF72 0.7D25 thermistor would be a better choice as it would waste up to six times less power. Still, losing a few watts on a 200w load seems reasonable if it saves me having to constantly replace DC power converters.
How quickly does the thermistor get hot? I got out my backup 54v DC power supply, and put one of these thermistors between the 54v supply and the 48-60v to 24v DC-DC converter. I then placed my battery load and capacity tester on the 24v output. It’s probably easier to show pictures of the test setup:
The power supply actually outputs 55.4v, and yes I did confirm this during testing. My battery load tester will only load up to 184 watts, so I firstly ran a test at 0.1 amps which is 2.4 watts on the output side. With conversion losses, that is at least 2.67 watts going through the thermistor:
With the thermistor at ~23 C, one should be getting around 2.5 ohms resistance so if the voltage drop is 0.198v and the current flowing is 0.048 amps, resistance should be 4.13 ohms. That’s obviously a lot more than expected, however at very low current flows the relative losses in the DC-DC converter will be high, so if say current flowing on the input side were actually 0.07 amps now you’re getting a resistance of 2.83 ohms and that feels more like it. If so, power lost to heat would be 0.014 watts, or 0.6% – this being so low because the thermistor does not radiate much heat into the environment.
Ramping it up to the maximum 180 watts which is about 200 watts on the thermistor:
The thermistor is now a toasty 130 C and the voltage drop is 0.66v. Current flowing through the thermistor should be around 3.61 amps, so power lost to heat would now be 2.38 watts, or about 1.2%. Resistance should be about 0.183 ohms, or about twice the resistance if the thermistor were at 200 C according to its datasheet.
When I first applied the 200w load, I noticed the voltage drop across the thermistor was momentarily 4.3v before rapidly falling within a second or two to less than 0.7v. If so, resistance was somewhere around the 1.2 ohm mark for whatever fraction of a second that was, and power lost to heat would have been 15.5 watts. No wonder that the thermistor gets hot so quickly!
If you leave the load tester off, the thermistor sits at about 20 C just over the ambient temperature. The parasitic draw of the unloaded DC-DC converter wastes very little heat in the thermistor. This is good to know. At a 100w draw which is probably around 110w for the thermistor, it reaches 92 C (I didn’t take a voltage drop).
Most of these DC-DC converters will be deployed into the LED strips for the outdoor lighting. The easiest solution is to run thinner wires to them which will offer the necessary protection against current surges: let’s say I fit 1.0 mm2 diameter wire, this would have a max amperage of 8-17 amps with a 4.24 volts per amp drop per 100 metres. 10 metres of such cable would offer 0.166 ohms of resistance, plenty more than the 0.1 ohms we need – so if I make sure I fit at least five meters of no thicker wire than 1.0 mm2 diameter, no more fuses will blow!
Modern fibre network testing with the Fluke LRAT-2000
Many years ago I picked up at vast cost a used Fluke LRAT-2000 which is an ethernet wiring validation tool. The LRAT-2000 was launched around year 2012, and it cost many thousands of euro – you could buy a used car for the price of a new LRAT-2000. Even used on eBay today fourteen years later, they still cost over a thousand euro which is quite remarkable given that all they do is (i) test each of the wires for being correctly connected (ii) helps you figure out which cable in a bundle is connected to which (iii) test that TCP traffic flows over the cable (iv) test PoE power loads. Why did I splash out such money like so many others on eBay still do for such legacy hardware? Simple: the quality is guaranteed to be there, so when it tells you that an ethernet cable has a break eighteen metres along, you can be confident that is where the break is before you take a kanga hammer to your wall.
One feature that it supports is fibre network testing. For this you need to insert a compatible fibre transceiver into the LRAT-2000’s SFP cage. As this is such old hardware, the fastest network supported is 1 Gbps and it turns out that a gigabit speed fibre transceiver in the legacy SFP format costs about €45 each even off Aliexpress, and of course you need two of them. €90 felt a bit steep for testing the eight or so fibre cables which are likely to be installed in my future house, so I looked for an alternative.
A few years ago I fitted 2.5G fibre transceivers as the backhaul between my Wifi 6 access points, these being the fastest possible for the legacy SFP (not the newer SFP+) cage – these were under €10 each, and they run with a relatively low power draw and very much tick the box for the problem they solve (distance backhaul). Years ago I bought a tray of those transceivers enough for all my future needs lest they fall out of production and become expensive, however of course the LRAT-2000 is too old to use those (it does recognise them if you plug one in, but it can’t find a network). What I therefore needed was some sort of box which let me connect the LRAT-2000 to a newer fibre transceiver, and it turns out that Aliexpress has exactly such an item: a cheap dumb media converter box which sends layer 2 network frames from one SFP+ cage to another. You then plug in whichever transceivers you want to have fibre to fibre, ethernet to fibre or ethernet to ethernet. There are also variants with a single SFP+ cage, and a RJ45 ethernet port.
(To be clear, you can also get branded media converter boxes from all the usual vendors e.g. TP-Link, but the unbranded models – which appear to be physically identical from pictures apart from a printed logo – are, or at least were before the tariffs, a good bit cheaper)
I ended up buying the following before the tariffs came in:
- Two 10G ethernet to SFP+ media converter boxes each for €38.33 inc VAT delivered.
- Two 10G single mode LC fibre transceivers for a fairly astonishing €7.71 inc VAT delivered. Yes, that’s a 5 km capable 1310nm wavelength ten gigabit transceiver for under a tenner.
And while this solution involves more wiring and power adapters than I would prefer, it does solve testing fibre connections with the LRAT-2000:
This was about the same cost as two of the legacy 1G fibre transceivers, so I didn’t save money – however I think this spend of ~€100 better bang for the buck.
High speed fibre transceivers have become ridiculously cheap in recent years: 40G fibre transceivers cost about €12 each, even 100G fibre transceivers are just €25 each. That makes 100G networking very feasible for a home network, which is just madness: the entire country of Ireland had 25G of international connectivity as late as year 2003. That’s 4x the entire former international connectivity of Ireland in your home!
What remains expensive is > 10G wired ethernet, as are > 10G switches. You can avoid expensive wired ethernet by fitting an inexpensive PCIe to SFP+ cage adapter to your PCs/servers and running fibre between everything, but a 10G switch with SFP+ cages is still a fair sum of money: Amazon currently lists an eight 10G ethernet port switch with SFP+ cage for €136 (albeit, it is managed not unmanaged). If you want faster than a 10G switch, used commercial switches are in the hundreds of euro range, but they are very loud and burn a lot of electricity; if you want something quiet and power efficient, it currently costs many thousands of euro. Given this, you may actually be better off building a cheap low power PC and filling it with PCIe to SFP+ cage adapters, however 40G ethernet SFP+ transceivers are not cheap nor do they sip power. One option is a switch of 40G SFP+ cages, these appear to be on sale used for about €500 right now, so you could probably build something somewhat quiet and power efficient for under a thousand euro if you populated that exclusively with 40G fibre transceivers. Still, that’s a thousand euro! Better than 10G networking remains a four figure cost for now.
And to finish, yet another bag of ESP32 dev boards …
If there is any iconic product line where Aliexpress has been absolutely stonking value for money it has been ESP32 dev boards. Two years ago I bought a bag of ESP32-C3 ‘Super Mini’ dev boards delivered and including VAT for €1.50 each. As described at the time, there were even cheaper dev boards on Aliexpress, the very decent RP2040 cost just €1.29 inc VAT delivered, but in terms of compatibility with a large existing microcontroller software ecosystem the ESP32 is hard to beat.
Before cheap Aliexpress went away forever, I therefore had a look around to see if any of the higher end ESP32 dev boards had come down to bargain basement prices, and indeed they had: there is now an ESP32-C6 ‘Super Mini’ dev board with most of the worst design mistakes in the C3 ‘Super Mini’ dev board fixed and those also come in bag sized quantities for tens of euro:
Comparing the C3 and C6 models directly:
| Board | Price I paid | CPUs | RAM | Flash | Wifi | Bluetooth | LR-WPAN | GPIO | Additional |
|---|---|---|---|---|---|---|---|---|---|
| ESP32-C3 Super Mini | €1.50 inc VAT in 2024 | 1x 160 Mhz RISC-V | 400 Kb | 4 Mb | v4 (2.4Ghz) | 5.0 | No | 13 | Blue LED |
| ESP32-C6 Super Mini | €2.74 inc VAT in 2026 | 1x 160 Mhz + 1x 20 Mhz RISC-V | 512 Kb | 4 Mb | v6 (2.4Ghz) | 5.3 | Yes (Zigbee, Thread etc) | 17 sides + 5 central | RGB LED, Battery charge |
The C6 is about 60% more expensive after adjusting for inflation. The C6 board is also available with a 8Mb flash variant (and it costs 4x more) which is useful if you want to run a Zigbee controller. But in essence, the C6 is the C3 except with a modern 2.4 Ghz radio hardware module implementing IEEE 802.15.4 which is a software defined wireless networking. This lets software easily implement many wireless networking protocols, such as Zigbee, Thread, Matter or anything similar. Wifi also gets modernised in the C3, going from Wifi 4 to Wifi 6, albeit still with only a 2.4 Ghz radio. And I suppose that the dev board does have 70% more i/o exposed, plus a lithium battery charge controller, if any of those things matter to you then the +60% cost may be worth it. Here is the C6 board right next to a C3 board for comparison:
For me, the main attraction of the C6 is for Zigbee routers (these extend a Zigbee network) and battery powered Zigbee end devices. Zigbee active consumes about 23-80 mA, and a Zigbee router needs to always be active, so one of my 3000 mAh lithium batteries would last only one hundred hours – still, if you had a Zigbee dead zone, firing in one of these with a bit of wire soldered onto its antenna to greatly extend range would bridge the issue, with low running costs when wired in for power.
Probably more exciting are battery powered Zigbee end devices e.g. a temperature sensor which could be dropped wherever you need them without having to wire them in. Zigbee end devices can deep sleep between periodic wakes to push data. Deep sleep on the C6 ‘Super Mini’ dev board can get down to 55 uA if you remove the RGB LED which always burns 330 uA even when off AND make sure you supply at least 3.6v to the battery pin/5v power input, otherwise the voltage buck converter burns current and it’ll horse through your battery quickly. If running off a battery, make sure to monitor the battery voltage and self disable if it gets below a certain minimum e.g. 3.4v, otherwise this board will happily run your battery down to damaging levels of empty.
The C6 has a proper low power 20 Mhz CPU for deciding whether to wake the board up or not which might be useful. The C3 has fixed function wake logic, so for example you might say ‘if pin A rises wake me’ which you can also do on the C6, but only wake the 20 Mhz CPU. It only has 16 kB of RAM accessible to it, but it CAN speak i2c and UART which is good enough to deal with a large subset of sensor boards. You could do fairly complex filtering or decision making with the 20 Mhz CPU, and wake the 160 Mhz CPU and wireless stack only if it were really necessary. This could shave off a few more percent of battery consumption.
According to real world numbers I researched from the internet, one of my 3000 mAh batteries fitted to a sleepy Zigbee end device and with a typical i2c sensor waking every ten minutes might last 10,000 hours = 1.14 years. When its battery gets low, you simply plug it in via USB and it’ll charge itself to full, then you can redeploy it. I can’t currently think of when I might use such a thing, but for the few euro to buy a bag of the C6s now before cheap Aliexpress ends it is definitely worth the hedge.
What’s next?
Most of this entry was written at the site while popups were installed. Expect a full write up and lots of pictures next entry!
Word count: 1446. Estimated reading time: 7 minutes.
- Summary:
- The input is a combined diary entry and a highly technical guide. A dark theme implementation is described, where pure CSS variables are utilised. Automatic switching based on the operating system’s theme can be achieved using
@media queries. Furthermore, a manual override is demonstrated via checkbox hacks, although significant code duplication must be accepted for the feature to be fully operational.
Thursday 18 June 2026: 07:40.
@media queries. Furthermore, a manual override is demonstrated via checkbox hacks, although significant code duplication must be accepted for the feature to be fully operational.
- Summary:
- The input is a combined diary entry and a highly technical guide. A dark theme implementation is described, where pure CSS variables are utilised. Automatic switching based on the operating system’s theme can be achieved using
@mediaqueries. Furthermore, a manual override is demonstrated via checkbox hacks, although significant code duplication must be accepted for the feature to be fully operational.
This post introduces the most radical style change to this website since the Hugo conversion back in March 2019: a dark theme which is automatically chosen by your web browser if your system’s theme is dark. If your system is configured to switch between light and dark themes based on time of day, so will this website. If you wish to override the current light-dark theme chosen, there is now a floating theme override button in the top right of the page. It works identically on mobile and desktop, and requires no Javascript: it is pure CSS only.
Major browsers gained this ability around 2020 and it works like this: Firstly, we replace all the colours in all the CSS with CSS variables, which have been available in major browsers from 2017 onwards:
/* light */
:root {
color-scheme: light dark;
--c-body-bg: #fff;
--c-body-fg: #000;
--c-navbar-bg: #fff;
--c-navbar-divider-bg: #000;
--c-hover-bg: rgba(0,0,0,0.1);
--c-hover-fg: #FF0000;
--c-hover-glow: #aa0000;
--c-tooltip-bg: beige;
--c-tooltip-fg: #000;
--c-tooltip-border: chocolate;
--c-tooltip-arrow: #000;
--c-pre-bg: rgba(0,0,0,0.05);
--c-quote-bg: rgba(0,0,0,0.05);
--c-shadow: #aaa;
--c-shadow-heavy: #000;
--c-affiliate-border: #000;
--c-post-details-border: #000;
--c-link: #00e;
--c-link-visited: rgba(85, 26, 139, 1.0);
--c-toggle-bg: rgba(255,255,255,0.33);
--c-toggle-fg: rgba(48,48,48,0.66);
--c-toggle-border: rgba(48,48,48,0.33);
--c-toggle-shadow: rgba(0,0,0,0.33);
--c-toggle-hover-bg: #fff;
--c-toggle-hover-fg: #333;
}
I actually had Step 3.7 Flash do most of this work, and it chose for the
CSS variable names a c- prefix for reasons I don’t precisely understand,
but equally it seemed a safer thing to do so I left it.
We now use the prefers-color-scheme CSS feature to tell the browser
to overwrite those variables when the system theme is dark:
/* Experimental CSS5 stuff */
@media (prefers-color-scheme: dark) {
/* dark */
:root {
--c-body-bg: #000;
--c-body-fg: #fff;
--c-navbar-bg: #000;
--c-navbar-divider-bg: #aaa;
--c-hover-bg: rgba(255,255,255,0.2);
--c-hover-fg: #FF0000;
--c-hover-glow: #ff6666;
--c-tooltip-bg: #2d2d2d;
--c-tooltip-fg: #d4d4d4;
--c-tooltip-border: #555555;
--c-tooltip-arrow: #555555;
--c-pre-bg: rgba(255,255,255,0.2);
--c-quote-bg:rgba(255,255,255,0.2);
--c-shadow: #333;
--c-shadow-heavy: #000;
--c-affiliate-border: #444444;
--c-post-details-border: #555555;
--c-link: lightskyblue;
--c-link-visited: #ad8bcd;
--c-toggle-bg: rgba(32,32,32,0.33);
--c-toggle-fg: rgba(240,240,240,0.66);
--c-toggle-border: rgba(240,240,240,0.33);
--c-toggle-shadow: rgba(255,255,255,0.33);
--c-toggle-hover-bg: #222;
--c-toggle-hover-fg: #eee;
}
Finally, you add to the <head> stanza to tell the browser that you support
automatic theme switching <meta name="color-scheme" content="light dark">,
and you will now have automatic theme switching, where the default theme is
light.
Making the theme user toggleable
Sometimes you might not want the theme to be dark or light depending on use case, so being able to override it manually is a must. You can do this trivially easy using four lines of Javascript, but I wanted to avoid Javascript so that unfortunately means stamping out quite a lot more CSS:
/* CSS-only dark mode toggle overrides via checkbox */
/* light */
#theme-toggle:not(:checked) ~ #page {
--c-body-bg: #fff;
--c-body-fg: #000;
--c-navbar-bg: #fff;
--c-navbar-divider-bg: #000;
--c-hover-bg: rgba(0,0,0,0.1);
--c-hover-fg: #FF0000;
--c-hover-glow: #aa0000;
--c-tooltip-bg: beige;
--c-tooltip-fg: #000;
--c-tooltip-border: chocolate;
--c-tooltip-arrow: #000;
--c-pre-bg: rgba(0,0,0,0.05);
--c-quote-bg: rgba(0,0,0,0.05);
--c-shadow: #aaa;
--c-shadow-heavy: #000;
--c-affiliate-border: #000;
--c-post-details-border: #000;
--c-link: #00e;
--c-link-visited: rgba(85, 26, 139, 1.0);
--c-toggle-bg: rgba(255,255,255,0.33);
--c-toggle-fg: rgba(48,48,48,0.66);
--c-toggle-border: rgba(48,48,48,0.33);
--c-toggle-shadow: rgba(0,0,0,0.33);
--c-toggle-hover-bg: #fff;
--c-toggle-hover-fg: #333;
}
/* dark */
#theme-toggle:checked ~ #page {
--c-body-bg: #000;
--c-body-fg: #eee;
--c-navbar-bg: #000;
--c-navbar-divider-bg: #aaa;
--c-hover-bg: rgba(255,255,255,0.2);
--c-hover-fg: #FF0000;
--c-hover-glow: #ff6666;
--c-tooltip-bg: #2d2d2d;
--c-tooltip-fg: #d4d4d4;
--c-tooltip-border: #555555;
--c-tooltip-arrow: #555555;
--c-pre-bg: rgba(255,255,255,0.2);
--c-quote-bg: rgba(255,255,255,0.2);
--c-shadow: #333;
--c-shadow-heavy: #000;
--c-affiliate-border: #444444;
--c-post-details-border: #555555;
--c-link: lightskyblue;
--c-link-visited: #ad8bcd;
--c-toggle-bg: rgba(32,32,32,0.33);
--c-toggle-fg: rgba(240,240,240,0.66);
--c-toggle-border: rgba(240,240,240,0.33);
--c-toggle-shadow: rgba(255,255,255,0.33);
--c-toggle-hover-bg: #222;
--c-toggle-hover-fg: #eee;
}
This is unfortunately very copy-and-paste, but I am unaware of doing better using pure CSS. In any case, we now overwrite those CSS variables based on whether the toggle theme checkbox is checked or not. Unfortunately we are not yet done with the copy-and-paste:
/* Experimental CSS5 stuff */
@media (prefers-color-scheme: dark) {
/* CSS-only dark mode toggle overrides via checkbox */
/* dark */
#theme-toggle:not(:checked) ~ #page {
--c-body-bg: #000;
--c-body-fg: #eee;
--c-navbar-bg: #000;
--c-navbar-divider-bg: #aaa;
--c-hover-bg: rgba(255,255,255,0.2);
--c-hover-fg: #FF0000;
--c-hover-glow: #ff6666;
--c-tooltip-bg: #2d2d2d;
--c-tooltip-fg: #d4d4d4;
--c-tooltip-border: #555555;
--c-tooltip-arrow: #555555;
--c-pre-bg: rgba(255,255,255,0.2);
--c-quote-bg: rgba(255,255,255,0.2);
--c-shadow: #333;
--c-shadow-heavy: #000;
--c-affiliate-border: #444444;
--c-post-details-border: #555555;
--c-link: lightskyblue;
--c-link-visited: #ad8bcd;
--c-toggle-bg: rgba(32,32,32,0.33);
--c-toggle-fg: rgba(240,240,240,0.66);
--c-toggle-border: rgba(240,240,240,0.33);
--c-toggle-shadow: rgba(255,255,255,0.33);
--c-toggle-hover-bg: #222;
--c-toggle-hover-fg: #eee;
}
/* light */
#theme-toggle:checked ~ #page {
--c-body-bg: #fff;
--c-body-fg: #000;
--c-navbar-bg: #fff;
--c-navbar-divider-bg: #000;
--c-hover-bg: rgba(0,0,0,0.1);
--c-hover-fg: #FF0000;
--c-hover-glow: #aa0000;
--c-tooltip-bg: beige;
--c-tooltip-fg: #000;
--c-tooltip-border: chocolate;
--c-tooltip-arrow: #000;
--c-pre-bg: rgba(0,0,0,0.05);
--c-quote-bg: rgba(0,0,0,0.05);
--c-shadow: #aaa;
--c-shadow-heavy: #000;
--c-affiliate-border: #000;
--c-post-details-border: #000;
--c-link: #00e;
--c-link-visited: rgba(85, 26, 139, 1.0);
--c-toggle-bg: rgba(255,255,255,0.33);
--c-toggle-fg: rgba(48,48,48,0.66);
--c-toggle-border: rgba(48,48,48,0.33);
--c-toggle-shadow: rgba(0,0,0,0.33);
--c-toggle-hover-bg: #fff;
--c-toggle-hover-fg: #333;
}
}
What we are doing here is flipping the handling of the toggle theme checkbox being ticked based on the system’s current theme i.e. you get the system’s current theme on page load, if the system theme changes the website also changes its theme, and the toggle theme checkbox being ticked therefore means ‘the opposite theme of the system theme’. Next we need a theme toggle:
<input type="checkbox" id="theme-toggle" class="theme-toggle" aria-label="Toggle dark mode">
<div id="page">
<label for="theme-toggle" class="theme-toggle-label" aria-label="Toggle dark mode">
<span class="icon-moon" aria-hidden="true">🌙</span>
<span class="icon-sun" aria-hidden="true">☀️</span>
</label>
...
</div>
Thanks to Unicode, the sun and the moon no longer need dedicated image assets like we would do in the bad old days. Nowadays, you just use the appropriate Unicode codepoint, dead simple.
The next thing is to make our toggle theme button NOT have a visible checkbox, and to style its label so it shows the moon or the sun and that floats at the top right with a transparency so it doesn’t get in the way of reading text:
/* Visually hidden checkbox but keyboard-accessible */
.theme-toggle {
position: fixed !important;
top: 0;
right: 0;
height: 1px;
width: 1px;
overflow: hidden;
clip: rect(1px, 1px, 1px, 1px);
}
/* Fixed floating dark-mode toggle button */
.theme-toggle-label {
position: fixed;
top: 1rem;
right: 1rem;
z-index: 9999;
width: 2rem;
height: 2rem;
border-radius: 50%;
background: var(--c-toggle-bg);
color: var(--c-toggle-fg);
border: 2px solid var(--c-toggle-border);
display: inline-flex;
align-items: center;
justify-content: center;
cursor: pointer;
font-size: 1.2rem;
line-height: 1;
box-shadow: 0px 0px 8px var(--c-toggle-shadow);
}
.theme-toggle-label:hover {
background: var(--c-toggle-hover-bg);
color: var(--c-toggle-hover-fg);
border-color: var(--c-hover-fg);
}
.icon-moon,
.icon-sun {
display: inline;
}
Finally, we need the toggle theme icon to be the correct one in all circumstances, which unfortunately requires yet more copy-and-paste CSS:
.theme-toggle:checked ~ #page .icon-moon {
display: none;
}
.theme-toggle:not(:checked) ~ #page .icon-sun {
display: none;
}
@media (prefers-color-scheme: dark) {
.theme-toggle:checked ~ #page .icon-moon {
display: inline;
}
.theme-toggle:not(:checked) ~ #page .icon-sun {
display: inline;
}
.theme-toggle:checked ~ #page .icon-sun {
display: none;
}
.theme-toggle:not(:checked) ~ #page .icon-moon {
display: none;
}
}
And that’s basically it – an unfortunate amount of duplication in the main CSS file, but it does get served to users gzip compressed so all that text duplication matters little to website load times. Most processors are so fast at parsing text that the extra verbiage doesn’t really matter either. As I mentioned above, you could use a few lines of Javascript and save yourself all this duplication, but I personally configure my web browser with Javascript disabled by default and I enable it on a per-site basis – only if I absolutely must use a site and it’s broken without Javascript do I enable it. If it’s a site I don’t care hugely about and it won’t load without Javascript I generally just close the tab, not worth the effort.
As mentioned above, Step 3.7 Flash did almost all of the work here. It didn’t get quite all the way there on its own though, because I wouldn’t let it install web page to image rendering via which it could debug its work. So I took it manually for the final stage of debugging and tweaking to get the last of it over the line. But I understand from reading online that if I had allowed it to visualise the results of its work and to click around the web page by itself, it should have successfully completed its task in full. The reason I didn’t give it the power to browse and interact with web pages by its itself alone is probably obvious: I hadn’t set up safety containerisation for it on my Mac, so it was effectively operating as if me.
It’ll probably be popups installation in my next diary entry. Depends on when they get it completed and what the weather will be like in the next few days. We shall see!