Healthy, Serving Traffic, and Missing 27 of Its 28 Tables

A founder built an application that keeps track of people's money. He built it himself, feature by feature, with AI coding tools, and he shipped it. It is live, people use it, and it does what he made it to do. He isn't named here, and neither is the product.

He then asked for something most people would not. He handed the whole thing over and asked us to take it apart from scratch, and agreed in advance that we could publish what came back, including the corrections we made to our own work along the way.

What follows is that account. It opens where the report opens, with what happened when we stood the application up on a machine of ours.

We built our own copy of the application, ran it on a machine of ours, pointed it at a brand-new database with nothing at all in it, and started it.

A database keeps its information in tables, the way a workbook keeps its information in sheets. This application uses 28 of them.

The application's own built-in status check reported healthy. The homepage loaded. The part that handles signing in answered normally. One of the 28 tables existed.

Those three are the checks anyone would run to confirm that a backup had been restored properly, and all three reported success against a database missing 27 of its 28 tables. Each one passes for its own reason. The status check never looks at the database. The sign-in service sees that nobody is signed in yet and stops there, before it needs to look anything up. The homepage is a fixed file sitting on disk, the same for everyone. Only a request that goes far enough to need a table shows that anything is wrong. And the one table that did exist is the one that records who is currently signed in, created by a piece of borrowed code, written by somebody else and dropped into the project, that sets itself up separately from the application's own start-up. The single survivor is the one table no feature depends on.

Six rounds of reading the code had predicted that failure. Twenty minutes of running it produced the part nobody predicted, which is that the system's own sign-in service will tell anything watching for trouble that everything is fine.

What he had built

The part of the application that runs on the server answers requests at 87 different addresses, and 61 of them belong to a game-like social side: groups you join with an invite code, leaderboards, a feed of posts including anonymous ones, private messages, head-to-head challenges, seasons that award medals, and a holding queue for posts that users have reported. The data splits the same way, 21 of the 28 tables on the social side. The README, the file that tells a newcomer what a project is, describes the smaller half. Anyone who agrees to keep this running has agreed to run a social platform that carries private messages and a queue of reported posts nobody currently reads.

The financial half occupies one table: three columns, one row per user, and a single block of data in each row holding all of it. That block is the whole state of one screen. Every time a user stops interacting for 1.2 seconds, the application writes the entire block back, every field in it, not only the field that changed. Inside are forty-six labelled values in one flat list, his users' finances beside their display preferences beside their game counters, with nothing separating them. There is no version number on the block and no step that brings an old one forward when the shape of it changes. Before storing it, the server checks that it is a block of data rather than something else, and that it is under half a megabyte. It does not check anything about what the block contains.

Nobody chose that. It is what a system settles into when nothing ever forces the decision to be named out loud.

What was already right

A survey that reports only faults is a false picture, so this part is specific.

When you sign in, the application issues you a fresh ticket that identifies your session, and it does the same when you register. That defeats an attack where somebody hands you a ticket they already know, waits for you to sign in with it, and then walks in as you. The defence is missing far more often than it is present, including in code written by people who do this for a living.

Passwords are never stored as passwords. Each one is run through bcrypt, a deliberately slow scrambling function, at a strength setting of 12, and the setting is the same in all four places in the code where a password is scrambled.

The push notifications carry bill names and amounts, and the borrowed code that sends them locks the contents so that only the recipient's own browser can unlock them. We confirmed that by reading that code ourselves rather than assuming it.

And the database is an exact match for the code. Twenty-eight tables with exactly the columns the code specifies, nothing left over on either side, no records pointing at things that no longer exist, and five running totals that the code keeps up to date by hand, all exactly correct. That is on a system which builds its entire database structure from its own code every time it starts, with no tool for managing changes to that structure and no version marker anywhere.

Where the bank statements go

When a user imports a bank statement, the whole file is sent off to an outside AI service to be read and turned into a list of transactions, along with the names the user has given their own bills and subscriptions. The PDF, the spreadsheet, the export, the screenshot, the pasted text. That is how the feature works, and it is a reasonable way to build it.

The code immediately around it is one of the better decisions in the application. Whatever the outside service sends back is forced into a fixed shape, date and description and amount, and anything that doesn't fit is thrown away before the user ever sees it. Most people take whatever such a service returns and write it straight into their own data. He did not, and that one decision is why "what if it sends back something strange" is a small question here instead of a serious one.

The decisions around that code did not get the same attention.

There is no disclosure of any kind. We searched the part that runs in the browser, and the project's README, for any mention of an outside company, any privacy statement, any step asking the user's permission. There is none. Nothing at upload time tells a user that their bank statement leaves the server.

The terms are unestablished. The password that opens the door to that outside service is stored as a setting on the server. How long that company keeps the files, who there can look at them, and whether any of it is used to train their models are all properties of whichever account that password belongs to, and that account is not ours. Until he has read his own account's terms, nobody can write an accurate disclosure, and a disclosure describing the wrong terms is worse than no disclosure. So this is a decision before it is a code change.

And the system cannot answer "whose statement went, and when." The code that handles an import never looks up which user is signed in. We checked that mechanically: no record kept per user, no limit set per user, nothing anywhere along that path that ties a file to the person who uploaded it. If a user asked him that question tomorrow, he could not answer it from the system.

The same kind of tool turns up twice in this account. It wrote the application, and it reads the statements. Which company sits behind either one would change nothing here. What is missing is a record of what left, on whose behalf, and when, and any word at all to the person it belonged to.

The examination

Nobody at BitSalt had written a line of this application. We started with nothing but the code itself and worked over two calendar days, split into nine turns, each taken by a different specialist: one working out what the software is supposed to do, one examining how it stores its data, one examining how the pieces fit together, one looking for security weaknesses, one deciding how it should be tested, and one looking at how it gets released and run. Each turn wrote down what it found while it was still working, fifteen entries in all, and every entry had to carry a section listing its own dead ends.

The standard is that no turn may begin until the one before it has produced its written result, and that result is reviewed before the next turn starts. What we got for working that way is below. Reading the code produced most of the report. The findings that mattered most came from running it.

One rule shaped all of it. This application holds real people's financial records and nobody here read one. We counted rows, listed the tables and their columns, timed how long things took, and read the code. Where answering a question would have meant looking at somebody's records, we left the question open and put it in the report's limits section, along with the check that would settle it.

Four things a careful reading would not have caught

A defence that has never run. When somebody tries to sign in under a username that doesn't exist, the application runs the submitted password through the same slow scrambling check anyway, against a fake stored result. The idea is that a guess at a real account and a guess at a made-up one should cost the same time, so nobody outside can tell the two apart. A note in the code says so.

The fake stored result is 58 characters long. A real one is always 60, and the code doing the comparison checks the length first and gives up before doing any of the slow work. Measured: about 0.02 milliseconds to answer "no such user," about 210 milliseconds to answer "wrong password." Anyone can tell which accounts are real, from outside, without signing in. The code reads as correct and the intent is written down twice. The only way to find it was to measure it, and nothing in the project could have, because the server side has no tests: no second set of code whose job is to check that the first set does what it claims.

A misplaced parenthesis. Money is rounded to the nearest cent the ordinary way: multiply by a hundred, round to a whole number, divide by a hundred again. On the line that adds a deposit to a savings goal, the closing parenthesis sits one term too early. Only the deposit gets multiplied, and then the whole sum gets divided by a hundred.

A goal holding $2,000 that receives a $50 deposit becomes $70, and that wrong figure is what gets written back as the goal's stored balance. Four other calculations prefer that stored balance as their source for total savings, so the health score and the net-worth history follow it down.

The same line appears three times in that file, on every route that adds to or subtracts from a goal's balance. The correct version of it sits in that same file, seventeen lines above the first mistake, and correctly a dozen more times elsewhere in the code, with nothing anywhere that would notice the two disagreeing.

Safety rules attached to the wrong replies. A web page can arrive with a set of instructions attached, telling the browser what it is and isn't allowed to do with that page. The application's server side sets a good default set of them, and it only ever handles data requests from the app itself. A second piece of software sits in front of it and hands out the actual page and the program the browser runs, attaching nothing but an instruction about how long to keep a copy.

So the safety rules ride along with data that no browser will ever display, and they are absent from the page a person actually sees. Three files, each defensible on its own, adding up to something none of them says.

The same join between those two pieces sets the limit on how large a file a user may upload. The application declares 10 megabytes. The software in front of it never declares anything, so its own built-in limit of 1 megabyte is the one in force. Two of our own documents and the code all agreed on the larger figure and were wrong together.

A file everyone had labelled stale. A file of database commands, sitting in the database folder, had been read, judged redundant, and written off as dead. By us, in the summary document handed to every turn that came after.

It runs every time the application starts, ahead of the application itself, and its second command hands administrator rights back to the lowest-numbered account each time. A new release goes out automatically whenever a change is sent up to where the code lives, so taking somebody's administrator rights away doesn't survive the next release. And if the lowest-numbered account is ever deleted, the rights land on whoever holds that position next, who never had them and was never told.

Ten minutes of searching for what actually runs that file turned the finding upside down. Reading a file's contents tells you whether they are redundant, which is a different claim from whether they run.

That same start-up line has one more thing to say. It runs the file under a time limit and then throws away the answer to "did that work," so a database step that failed and one that succeeded look identical to anything watching. The wrapper is ours, out of a separate five-day engagement in April that set up this application's release process and touched none of its own code, before this standard existed. It belongs in this list rather than outside it.

Most of it is not happening

The system has a handful of accounts and a social half that is essentially unused: no groups, no private messages, no push-notification sign-ups. We counted the rows in sixteen of the tables, and seven of those sixteen came back completely empty. A flaw in the way private groups keep themselves private costs nothing when no groups exist. A cap on incoming requests that gives the whole user base one shared allowance instead of one allowance each is not an outage at this size.

So every finding carries two ratings that move independently, what it would cost if it happened and how close it is to happening, and the list is ordered by the second. The restore failure is the one exception, and it leads, because its cost doesn't move with the number of users. A handful of people's financial records are exactly as unrecoverable as several hundred would be. Getting from a restored backup to a working system takes four to six manual steps that are written down nowhere. And every new release overwrites the last one under the same label, with no older copy kept beside it, so going back to the previous version after a bad release isn't an operation that exists.

Whether the savings defect has taken money off anybody is unmeasured. A user can reach it, the wrong value gets stored, and the balances sit inside the block of data we did not read. At most it is a handful of people and it could be nobody, and guessing would have been worth less than saying that.

Three things this examination changed about how we work

Every entry in the record carries a section on its own dead ends, written while that turn was still going. Three of those changed how the examination ran, and each one is why something in this account works the way it does.

Every finding carries two ratings, and the order follows the second one. We arrived at that design by getting it wrong twice.

First, fourteen gaps in who is allowed to do what were ranked, and five questions raised alongside them, several urgent in tone, before anybody had established how many people use the system. That count was the cheapest question available, and calling a problem severe without saying how many people it reaches implies a crowd of users nobody has counted.

Then the findings were sorted by severity alone. That put a flaw in the way private groups keep themselves private, attached to zero groups, above a defect that destroys a figure the user typed in and sits one ordinary action away. Producing two independent ratings and then sorting on one of them throws the second one away. The order in this account is the result.

The data rule is enforced on language, not only on actions. A draft note to the client said we could see how many accounts there were. Our reviewer changed see to count. Every turn had honoured the rule in practice: the number came from counting rows, and no row's contents were ever returned. The first sentence written to leave the building got it wrong anyway. A rule enforced on actions and not on language is half a rule, and language is all a reader gets.

Quoting a source is not the same as checking it. One entry in the record listed the row counts as prose: a table with zero rows, then a run of small numbers. One post, one comment, one friendship, four notifications, nine invites, one session. The nine was the invite count. A later turn read that nine, attached it to the wrong word beside it, and wrote that nine of the 28 tables were empty. Seven came back empty, out of the sixteen tables the count actually covered. The sentence went into the client's report, the security review and the test plan. Every one of them quoted the record entry, and quoted it correctly. Not one of them opened the raw output the entry was summarising.

That is the same error this account is about, committed by us, one document layer up. Reading a file tells you what it says. Only running the query tells you what is true. It was caught because one turn, whose whole job was checking facts, had been told to check them against raw output rather than against the documents quoting it. That is the difference between an examination that reads and one that checks.

The record carries eight corrections to our own work, and three of them are the same error: a claim asserted without checking what would prove it wrong. The standard doesn't stop them. It caught every one, each by something specific. Three by our reviewer. Two by a later turn contradicting the summary it had been handed. One by checking a claim before passing it on. One by an automated rule that refused to save a change. One by the fact-checking turn reading the raw source instead of the documents quoting it. None by anyone being careful.

What this says about software built this way

Nothing found here is a mistake a tool made in the moment. Every one of them looks right if you only look at it. The code that reads the bill-payment record back out reads exactly as it would if the code that saved it had saved what this code expects. The savings calculation is one character from correct and sits inside a pattern used correctly a dozen times in the same body of code. Care within a single change is not what is missing.

What is missing is anything that would notice a change contradicting an earlier one. Nothing writes down what a record is supposed to contain. Nothing declares what kind of value each field holds. There are no tests on the server side, no step that carries old data forward when the shape of it changes, and nothing that compares the pieces and reports a disagreement. Two people writing this code months apart would have collided immediately. One person writing it months apart, with a tool that produces a locally correct answer every time it is asked, never collides with anything.

For a founder holding one of these, "does it work" is settled, and users settled it. The open questions are what happens on the first day something has to be restored, and what happens in the first week the empty half stops being empty. Both are answerable now, in advance, by somebody other than him.

The system is still running and nothing about it has been changed. Every claim in the report we handed him names the file and line it came from, or names the check that would settle it, so the whole document can be walked line by line with the source open beside it. That was the deliverable: something its author can argue with, and then decide about himself.