18 October 2026

Eighteen entries today:

• A user's _ properties are never written to userdetails
• schema:diff on an installation that adopted its tables, and foreign keys looked for in their own database
• db:migrate-from-mysql: an application's data from MySQL into PostgreSQL
• db:migrate-from-mysql: a live move in three phases — operational, cutover, backfill
• db:migrate-from-mysql --transport=fdw, and the test image with mysql_fdw
• db:migrate-from-mysql from MariaDB
• db:migrate-from-mysql --source-timezone: a DATETIME read in the zone that wrote it
• db:migrate-from-mysql: a table it makes gets MySQL's indexes and foreign keys
• db:migrate-from-mysql: verify, a cutover that compares, composite keys, deletions in ranges, no snapshot
• The rate limit's token bucket survives a clock that steps back
• The query builder prefixes a bare table name, as the schema builder does
• /health/check keeps its findings for administrators; the client_credentials iss is the issuer
• db:migrate-from-mysql: the source read in utf8mb4, double-encoded text found and repaired
• db:migrate-from-mysql into identity and computed columns
• db:migrate-from-mysql: the dry run names what compared without regard to case
• db:migrate-from-mysql: hypertables verified window by window, and each backfill window checked
• db:migrate-from-mysql --phase=operational takes up a copy that stopped
• db:migrate-from-mysql --batch-mb: batches bounded by bytes

A user's _ properties are never written to userdetails

FW-155.

Fixed

• User::save() wrote a _-prefixed scalar that had not been loaded.
  • The guard against _ fields sat on the second half of an ||, because && binds first. So a private property that reached __set() without being loaded was written on every save: a subclass's cache of a serialized model, cut at 255 characters.
  • One installation found 262,000 such rows.
  • The condition is now parenthesised. A characterization test on MySQL and PostgreSQL holds that neither a scalar nor an array _ field becomes a row, while a plain one still does.

Documentation

• The Authentication Guide says which properties become userdetails rows.

schema:diff on an installation that adopted its tables, and foreign keys looked for in their own database

FW-156.

Fixed

• Two migrations looked for a foreign key in every database on the MySQL server.
  • add_missing_foreign_keys_to_existing_tables and repair_gdpr_requests_and_authserver_foreign_keys asked INFORMATION_SCHEMA by table and constraint name only.
  • So a same-named key in any other database on the server read as present here: a schema:diff reference, a copy. migrate then recorded the repair as run without adding its keys.
  • Both now ask in DATABASE().
• schema:diff's reference left out migrations.exclude. On an installation that adopted its tables, those are the CREATEs of exactly the adopted ones. They were never compared, and the migrations that alter them failed inside the reference.
  • The reference now builds them by default. --honour-exclude leaves them out.

Added

• schema:diff --fresh empties a named --reference first. A named reference that is not empty is refused without it.
• schema:diff checks the database user's privileges up front. It tries CREATE, ALTER, INDEX and DROP on a probe table before any migration, and names the first it may not run.
• schema:diff names an allowance that matches nothing after the report.

Documentation

• The Migration Guide's schema:diff section describes the above.
• Two pages named a private application in an example. They now use neutral wording.

db:migrate-from-mysql: an application's data from MySQL into PostgreSQL

Added

• db:migrate-from-mysql --phase=full moves an application's data from MySQL into the PostgreSQL (or TimescaleDB) database its migrations built.
  • Each value is converted to what its target column takes: flags to booleans, unix times to moments, zero dates to NULL, JSON checked, bytes to bytea, text made valid UTF-8.
  • A value that had to become NULL is counted.
  • MySQL's authserver_x goes into authserver.x.
  • A table no migration makes is created from its MySQL definition.
• How the rows load.
  • Everything is read in one consistent snapshot and streamed.
  • Rows go in batches with ON CONFLICT DO NOTHING, so a second run adds only what is missing. A refused batch is retried row by row.
  • Tables load parents first. Keys are off where the target user may turn them off, and the orphans that leaves are then counted (--null-orphans sets them to NULL).
  • Sequences are moved past the highest id.
• --dry-run names what would not arrive: columns only MySQL has, columns only the target requires, compressed target chunks.
• Database\FromMysql\Migrator, TypeMapper and TopologicalSorter are the pieces.
• The operational, cutover and backfill phases for a live production move come next.

Documentation

• A new guide, Moving from MySQL to PostgreSQL. The Console Guide lists the command.

db:migrate-from-mysql: a live move in three phases — operational, cutover, backfill

Added

• --phase=operational copies the tables while the application runs. It records the snapshot's moment and each table's highest id in pramnos_migration_progress on the target.
• --phase=cutover runs with writes stopped and moves what changed since the copy, written over the target.
  • New rows are found by id, changed rows by their change column (updated_at and the like, or --changed-column). With --with-deletes, rows deleted in MySQL are deleted here too.
  • A table whose changes cannot be seen is named.
• --phase=backfill moves history window by window, newest first, up to the cutover's moment.
  • Each finished window is remembered, so a stopped run continues; --restart starts over.
  • On a hypertable, the compression policy is removed for the backfill and put back after, and compressed chunks in a window are decompressed first.
  • --compress-after-chunk compresses each window as it finishes, and --throttle-ms paces the batches.
• Database\FromMysql\ProgressTracker keeps the progress.

Documentation

• The guide, Moving from MySQL to PostgreSQL, has a section on the three phases.

db:migrate-from-mysql --transport=fdw, and the test image with mysql_fdw

Added

• --transport=fdw moves rows through mysql_fdw on the PostgreSQL server, as one INSERT … SELECT per table or backfill window, with the values converted in SQL.
  • A table or window that SQL cannot convert (bytes that are not UTF-8, a NUL byte) moves through PHP instead, and the report says so.
  • The foreign server, mapping and tables exist only for the run. --fdw-host names MySQL as the PostgreSQL server reaches it.
• Database\FromMysql\FdwTransport makes the foreign tables and the SQL conversions.

Changed

• The test TimescaleDB is built, from docker/timescaledb/Dockerfile: the same pinned timescale/timescaledb:2.26.4-pg17 with mysql_fdw 2.9.2 compiled in.
  • The extension is made available and never created, so nothing else in the suite changes.
  • Run docker-compose up -d --build timescaledb once to get it.

Documentation

• The guide, Moving from MySQL to PostgreSQL, has a section on mysql_fdw.

db:migrate-from-mysql from MariaDB

Added

• The command's tests run against MariaDB 10.11 as well as MySQL 8. MigrateFromMariadbTest and MigrateFromMariadbPhasesTest extend the MySQL ones and run with ./dockertest --mariadb. Every assertion holds on both: the full move, the three phases and both transports.

Fixed

• A table the command created from MariaDB lost its JSON and its defaults.
  • MariaDB reports a JSON column as longtext, so the created column was text. The json_valid() check that marks it is now read, and the column is created as jsonb.
  • MariaDB spells defaults its own way: a quoted literal, current_timestamp(), NULL for none. These are now read as what they mean.
  • A created column keeps a literal default as well as a number or the current time. A zero date and an expression with no PostgreSQL equivalent are left out.

db:migrate-from-mysql --source-timezone: a DATETIME read in the zone that wrote it

Added

• --source-timezone names the zone the source's DATETIME values were written in. It defaults to the application's zone.
• The dry run lists every DATETIME that goes into a timestamptz. It warns when the MySQL server runs at a different offset from that zone.

Fixed

• A DATETIME going into a timestamptz was read in PostgreSQL's session zone. Every value could shift by the offset between the two servers. It is now read in --source-timezone, by both transports.
• A MySQL TIMESTAMP arrived in the reading session's zone. The source is now read with the session in UTC, as mysql_fdw reads it, so the PHP and fdw transports agree, and a TIMESTAMP is the moment it is.
• The cutover compared a DATETIME change column with the source server's clock. The backfill cut its windows by that clock too. Both now use the zone the values were written in. For a zone behind UTC, the cutover had missed every change of the last hours.
• A date that is not one is now counted and written as NULL, rather than failing its batch.

db:migrate-from-mysql: a table it makes gets MySQL's indexes and foreign keys

Added

• A table no migration makes now gets more than its primary key. Once its rows are in, it gets MySQL's indexes, unique keys and foreign keys.
  • Foreign keys keep their ON DELETE / ON UPDATE rules, and point at wherever the referenced table went.
  • The table loads after the tables it points at.
  • A key its rows break is added NOT VALID and named, with the VALIDATE CONSTRAINT to run later. With --null-orphans it is validated after the references are cleared.
  • A prefix index covers the whole column.
  • FULLTEXT, SPATIAL and expression indexes are named in the dry run, not made.
  • A run that stops after making the table and before its indexes leaves it marked in pramnos_migration_progress. The next run finds it unfinished and finishes it. A table the migrations made carries no mark and is left alone.

db:migrate-from-mysql: verify, a cutover that compares, composite keys, deletions in ranges, no snapshot

All of it rests on one new piece, Pramnos\Database\FromMysql\KeyRanges. It walks a table by its primary key a range at a time (WHERE (key) > (last) ORDER BY key LIMIT n), on either server and for composite keys too. It compares rows as the target holds them: each source row is converted, cast to the target's column types, and hashed by PostgreSQL beside the target's row.

Added

• --phase=verify, and --verify after full or cutover.
  • Counts each table on both sides.
  • Compares every row by key.
  • Looks for rows only the target has.
  • Names up to five keys per table and exits 1 on any difference.
• --no-snapshot for full and operational: each table is read a range of keys at a time, so no transaction is held open on a busy source for the hours a copy takes.
• --append-only for cutover: a big table that only grows is read for its new rows alone.

Changed

• The cutover of a table without a change column compares every row with the target and writes what is missing or differs. It used to take only the rows past the highest id, and rows that existed at the copy and changed since were lost to the move.
• The cutover works with a composite primary key. Changes are read by change column or by comparison, and are written ON CONFLICT on the whole key. Such a table used to be named and left.

Fixed

• --with-deletes read every key of a table into memory, on both sides. It now walks the target a range of keys at a time and asks MySQL which of those it still has. Memory stays at one range. A key whose text differs between the servers is named, and nothing in that table is deleted on a guess.
• An orphan in a column that takes no NULL was reported with "--null-orphans sets them to NULL", which it cannot do. Such orphans are now named as needing to be deleted, or the row they name added.

The rate limit's token bucket survives a clock that steps back

Fixed

• A clock stepped back drained the bucket. The time since the last take was used as it came, so a negative one (NTP correcting the clock, a virtual machine resumed) took tokens away. An hour back left a full bucket thousands of tokens below zero, and its client was refused for that hour. Both stores, the Redis script and the cache path, now count it as no time passed.
• TokenBucketTest failed now and then in a full run. It checked that a token was spent at a refill rate of ten a second, so a pause of a tenth of a second between two takes had already refilled it. It now runs at one a second.

The query builder prefixes a bare table name, as the schema builder does

Changed

• table('things') reads {prefix}things on an installation with a prefix. That is the table createTable('things') made. The two builders had disagreed: the schema builder prefixed a bare name, the query builder left it.
  • A query on a bare name read a table no migration had made, or another application's table of that name in the same database. A test once inserted into the real tables that way; only their NOT NULL columns refused the rows.
  • In a SELECT the prefixed table is aliased by its bare name, so things.id still names it. Writes name it without an alias.
  • #PREFIX#, already-prefixed names, schema.table, pg_*, subqueries and the query's own CTEs are left as written. The prefix is applied when the statement is compiled, so a CTE defined after the from() that names it is still recognised.
• 'database' => ['prefix_bare_names' => false] keeps bare names as written, for an installation whose own tables have no prefix.

What breaks

On an installation with a prefix, a query on a table the application created itself without the prefix now looks for the prefixed one. Turn prefix_bare_names off, or rename the tables. The Upgrade Guide has the row.

/health/check keeps its findings for administrators; the client_credentials iss is the issuer

Changed

• /health/check tells an unsigned caller what a monitor needs, and no more. That is the verdict, its 200/503, and each check by name as ok or error.
  • What each check found (its message, and details naming versions, drivers, hosts and latencies) goes to a signed-in administrator.
  • With 'health' => ['public_details' => true] it goes to everybody. That is the switch the authorization server's discovery health already used.
  • The endpoint stays public, because an uptime monitor calls it unsigned.
• A client_credentials access token's iss is Discovery::issuer(). That is sURL with its trailing slash, the discovery document's issuer. It had been the URL without the slash, so a client comparing the two, as RFC 8414 has it do, found them different. A token is resolved by its usertokens row and not by iss, so tokens already issued keep working.

What breaks

A monitor that read a check's message or details from /health/check without signing in, and a client that compared iss with the slash-less URL. The Upgrade Guide has both rows.

db:migrate-from-mysql: the source read in utf8mb4, double-encoded text found and repaired

Added

• The dry run reads a sample of each table (up to 2,000 rows) and names:
  • every column holding double-encoded text (UTF-8 written through a latin1 connection, so Ελλάδα reads Ελλάδα), with one value as it would be repaired;
  • every text column whose MySQL character set is not UTF-8.
• --repair-encoding=table.column,… repairs those values on the way.
  • A value is replaced only when, turned back into Windows-1252 bytes, it is valid UTF-8 that differs from it. Genuine latin1 text and proper Greek are left alone.
  • --phase=verify with the same option compares the values as repaired.

Fixed

• The source connection's character set was the client library's default. Here that is utf8mb4, but nothing guaranteed it. A latin1 connection would have read every Greek letter as two characters, and nothing would have refused them. The migrator now sets utf8mb4 on the connection itself.
• No test of the command moved anything but ASCII. One now moves Greek, an emoji, a genuine latin1 column and a double-encoded one, from MySQL and MariaDB, through both transports.

db:migrate-from-mysql into identity and computed columns

Fixed

• A target key made GENERATED ALWAYS AS IDENTITY refused every row. PostgreSQL refuses a value for such a column unless the insert says OVERRIDING SYSTEM VALUE. Both transports now say it, so the moved ids are kept and the identity continues past them.
• A column the target computes (GENERATED ALWAYS AS (…) STORED) refused every row that carried a value for it. It is now left out of the move, named in the dry run, and computed by PostgreSQL.

db:migrate-from-mysql: the dry run names what compared without regard to case

Added

• The dry run names every indexed text column that MySQL compares without regard to case (a _ci collation) and PostgreSQL will compare exactly. That is where a login by e-mail stops matching a different spelling, and where a unique key starts taking A beside a.
  • It suggests citext, or lower() in the index and the query.
  • A citext target column and a binary collation are not named.

db:migrate-from-mysql: hypertables verified window by window, and each backfill window checked

Added

• --phase=verify --time-column=… compares a table window by window. Each window's rows are counted and digested on both sides, and the windows that differ are named.
  • The digest is the sum of a 32-bit hash of every row as the move writes it. It does not depend on row order, so it needs no key.
  • This is how a hypertable is verified. Its rows arrive without MySQL's id, and the key comparison could only count them.
• The backfill checks each window as it writes it. It counts the window's rows on both sides and names a window that lost rows at once, failing the run. With --verify it digests each window too.

db:migrate-from-mysql --phase=operational takes up a copy that stopped

Added

• A second run of a stopped operational copy continues it.
  • Tables already finished are not read again.
  • The table being copied when it stopped is read on past the highest id already in the target.
  • That table keeps the moment and highest id of the attempt that began it. The cutover then reads changes from the earlier moment, and nothing written between the two attempts is missed.
• --restart copies every table again.

A copy of ten hours that stopped at the fortieth table used to start over from the first.

db:migrate-from-mysql --batch-mb: batches bounded by bytes

Added

• --batch-mb (default 32) bounds a batch by the bytes of its values as well as by --batch-size rows. This applies to inserting, comparing and digesting.

Fixed

• A table of large BLOBs filled PHP's memory. A batch counted only rows, so 5,000 rows of a few megabytes each were held at once.
• A read by key range sized its first range before knowing anything. It asked for the whole batch. It now reads one row to weigh, and sizes each next range by what the last one weighed.